Thoughts

status update 2

Published on

a few things to go over.

the rbqc and hard exclusions

the rbqc pipeline is intuitively set up. it's not hardly backed by any peer-reviewed science, but i think it's solid enough to at least make hard indications of the "red" wells. as of today, which is day 16, red has been triggered on all of col 12, and 3 non col 12 have been marked as attrition and have been excluded from downstream imaging + upstream segmentation. so we are left with 85 EBs, and just today H8 has been shrinking to the point that that might be also removed as well. H11 isn't looking much better either. it should be dangerously orange, which i will bring up when reporting the holistics to b.

if both removed, we will be left w/ 83 brave soldiers for the ongoing anabasis.

i mentioned that the pipeline is intuitively set up because it's not backed by science. a good thing to bring up in the meeting, separately of preservative ground truth collection, is coming up with scientific thresholds; if there is none, still consult for their intuitive opinions and make threshold changes accordingly. if we all align with the current thresholds, leave them as they already are.

out of frame, inertia & moving onto 4x 24w

the out of frame problem seems semi solvable from the inertia cheat. that is, to perhaps spin around clockwise to gradually "lower" the directional position of the EBs under plate reader. it cannot be done too much. everything should be in moderation. for example today i saw everything collectively went down after i did some spinning of my body with the EBs, causing everything to be in frame except for A2, which is now touching the down border. but again, it's not a big deal if i don't do it often, or if i do it, not do it too hard.

this is not a long term solution, and would vanish as a problem entirely after the transition to 24w plates. and we are 6 days away from that. as for the access for the new machine, that is not fully set but b is on it. worst case scenario is doing manual imaging on sunday. and i have all the time on sunday. just not after sunday.

intuition on covariates & modeling independency vs. dependency

covariates, in my understanding, is essentially a flag that tells the model "there's moderate indications of positional artifacts' existance for this set of EBs, but they are mild enough to be included, and if they are alive by the time of IHC, what we will do is to designate the responsibility of inspection to ENR itself, not by humans. we will give each EB flagged for covariate an unnormalized 'feature' of digit 1 for the ENR to pay more attention to it and decide the amount of attention it needs for these special cases." tl;dr, it is the insurance to help boost generalizability.

the hard qc rules is modeling independent. the hard qc rules certainly have higher gates to pass in order to be flagged. but for the mild, seemingly positional problems should be flagged as well, just at a lower, more nuanced standard. i wrestled with the necessity and method of implementing this until i've realized that the gates, however statistically rigorous (which it currently is), needs to fulfill two more things:

  1. covariates are modeling-dependent, meaning that they shouldn't be implemented within datasets prior to ground truth collection AND modeling. that is because the temporal window of the ENR-ready dataset and also the covariate setup has to be identical to have all the covariate calculations temporally align. gate 1 and 3 look at last day for all metrics, and it gets harder to make a positive verdict (that might be false positive) by using mann-whitney U. gate 2 looks at the full area window progressively and identify the degree of widening between each row/column to every other row/column.

  2. the dilemma of the overperformance inclusion for covariate calculation. this is actually quite easy when looking from the macro scale, that again, this is modeling dependent. the ultimate goal for this insurance of a covariance protocol is to boost generalizability (as i've mentioned previously), so whether to include overperformance (the original rationale being it is still positional or abnormal) or exclude (the original rationale being the bionormality of growth excellence) ultimately boils down to the true rationale of boosting generalizability. so it is down to the performance. and the performance of each decides whether the inclusion or exclusion is actually better.

  3. note for number 2: although the dilemma is going to be resolved down the line, looking at gate 3 of the calculation does require biological understanding of circularity and eccentricity at various temporal windows (first numbered list discussed this). it is not so simple as in area (bigger is generally better, vice versa) to understand what is good and what is bad in terms of the circularity and eccentricity deviations. therefore, it might actually be more generalizable (this is intuition, not through reasoning) if we do include all deviations of all features.

  4. note for the note: as claude code's research came to an end about this matter, there is no consensus on what is good vs. bad eccentricity, but circularity is generally agreed to be good when high early, low later -- but this is also protocol dependent, so test 3 has to be all deviations. and for the overall decision tree of number 2, the inclusion of overperformance can only certainly be interpreting area and area only, which only deals with gate 1 (conerning area), not gate 2 (also concerning area but gate 1 already does the job and it's an "AND" situation not "OR" situation for the screening) gate 3 (concerning every other feature). this has been noted and confirmed within the current covariate screening pipeline.

the meeting on ground truth

it is probably important to at least go through the key ideas of made to stick so that the progress update is easy to understand. to convey ideas more concseily is always better than complexity.

as for indivualized or trackability of ground truth collection, claude code gave some potential industry-standard solutions, namely:

  • TMA-style grid molds (most practical option)
    • Embed organoids into a mold with a defined grid layout (e.g., a 5-position mold where position 1 = A1, position 2 = A2, etc.)
    • The spatial arrangement is preserved through sectioning, so you read identity from position
    • Commercial options: Arrayit, Tissue-Tek TMA molds, or 3D-printed custom molds (increasingly common in organoid labs)
    • You'd process one row/group per block with a recorded position map
  • other approaches
    • Tissue marking dyes (e.g., Davidson Marking System) — colored dyes applied before embedding that survive fixation/processing. Limited to ~7 distinct colors, so you'd combine with positional encoding
    • Color-coded agarose pre-embedding — embed each organoid in a small agarose bead doped with different fluorescent dyes before putting them in the block
    • Barcoded microparticles — co-embed tiny barcoded beads with each organoid (more cutting-edge, less routine)

to quote directly from cc: "For your case specifically (93 wells, 5 per block = ~19 blocks): a grid mold with a position map is probably the simplest. [b] would place organoids in defined positions within each block and you'd maintain a spreadsheet mapping block ID + position → well ID. No fancy reagents needed, just discipline during embedding. Worth asking s if the lab already has TMA molds or if a custom 3D-printed mold would be feasible for the number of organoids you're working with."

that is all.

anyways, the anabasis continues.