Show code
from google.colab import drive
drive.mount('/content/drive')Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
Two readouts are analysed side by side in one notebook:
| Assay | Plates | Raw measurement | Line style |
|---|---|---|---|
| Castor | 1, 2 | image-based confluence (fraction) | dashed |
| MTT | 1, 2 | MTT absorbance (stored in the same Confluence column) |
solid |
Plate numbers repeat across assays — Castor P1 and MTT P1 are different physical plates, so every readout file, every layout and every grouping below is keyed on (Assay, Plate), never on the plate number alone.
File naming this notebook expects in DATA_DIR:
<ASSAY>-<anything>_P<plate>_D<day>.csv (e.g. CASTOR-…_P1_D7.csv, MTT-…_P1_D7.csv)Plate<plate>_layout_<Assay>.csv (an assay-less Plate<plate>_layout.csv is applied to every assay)The same drug measured both ways stays two separate series: normalization, the reversal filter, LL.4 fits and every IC50 are computed within an assay.
Plot contract: colour = drug (identical across assays), linestyle/marker = assay.
Raw metric: Day 7 only — no Day-0 normalization. Responses are normalized to the untreated wells of the same assay/plate/drug (%Viable), which also puts the two assays’ different raw units on one comparable scale. IC50 computed two ways: fitted LL.4 plateau midpoint and absolute 50% of untreated by direct interpolation.
from google.colab import drive
drive.mount('/content/drive')Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
# ====================================================================================
# USER INPUT — Navigate to experiment folder shortcut; copy relative path to DATA_DIR
# ====================================================================================
DATA_DIR = '/content/drive/MyDrive/20260827 Cytotox - Hu64_VPE6_GE6/Capan Combine Analysis-20260910/Data'from IPython.display import display, Markdown, Image as IPImage
import os
import re
import pandas as pd
import numpy as np
from scipy.optimize import curve_fit
import matplotlib
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import Patch
from matplotlib.legend_handler import HandlerTuple
import seaborn as sns
import warnings
warnings.filterwarnings('ignore')
sns.set_style('whitegrid')
plt.rcParams.update({'figure.figsize': (10, 6), 'font.size': 12})
# =====================================================================
# USER INPUT — change only this section to run a different experiment
# =====================================================================
CELL_LINE = 'Capan' # cell line name shown in plot titles
TEST_AB = 'Hu63' # test article antibody prefix
ISO_AB = 'Iso' # isotype control antibody prefix
R2_THRESHOLD = 0.5 # minimum R² for a "reliable" LL.4 fit
SCALE_MIN_CONC = 0 # dose-response x-axis lower bound (0 = untreated pseudo-conc)
SCALE_MAX_CONC = None # None uses all concentrations retained by the reversal filter
SCALE_EXCL_ABS = [] # all retained drugs contribute to y-axis scaling
REVERSAL_FILTER_METRIC = 'Confluence' # raw metric used to detect fragmentation artefacts
REVERSAL_INCREASE_THRESHOLD = 0.25 # exclude if >25% above the preceding concentration
# --- USER INPUT: assay merge (multi-readout experiments) -------------------
# Each readout CSV is tagged with an assay label so the same drug measured by two
# different readouts stays two separate series and is never pooled. Plate numbers
# may repeat across assays (Castor P1 and MTT P1 are different plates), so every
# grouping downstream is keyed on (Assay, Plate).
#
# readout data -> <ASSAY>-<anything>_P<plate>_D<day>.csv
# plate layouts -> Plate<plate>_layout_<Assay>.csv
# (assay-less Plate<plate>_layout.csv applies to every assay)
#
# Assay resolution per readout file: FILE_ASSAY -> ASSAY_PATTERNS -> DEFAULT_ASSAY.
ASSAY_PATTERNS = [
(r'^CASTOR[-_ ]', 'Castor'), # Castor image cell counting
(r'^MTT[-_ ]', 'MTT'), # MTT chemistry
]
FILE_ASSAY = {} # per-file override, e.g. {'oddly_named_P1_D7.csv': 'MTT'}
DEFAULT_ASSAY = 'Castor' # for files matching no pattern and no override
# Plot style contract: COLOUR = drug (identical across assays),
# LINESTYLE / MARKER / HATCH = assay.
# linestyle/linewidth/marker are the on-plot cue; hatch is the bar-chart cue.
# Castor's dash is (0, (6, 2)) rather than '--': long enough to stay obvious
# where curves overlap, and in a short legend sample.
ASSAY_STYLES = {
'Castor': {'linestyle': (0, (6, 2)), 'linewidth': 2.2, 'marker': 'o', 'hatch': '//'},
'MTT': {'linestyle': '-', 'linewidth': 1.7, 'marker': 's', 'hatch': ''},
}
# Plots of the RAW metric (growth kinetics, dose-response) show quantities in
# different units — Castor confluence vs MTT absorbance. Sharing one y-axis
# pushes them into separate bands, so only their x-positions can be compared.
# The assay named here is drawn against a twin right-hand y-axis on its own
# scale: each readout then fills the full plot height and the two sets of
# points/curves roughly overlap, comparable by shape. Presentation only — fits,
# IC50s and %Viable are computed per assay and are unaffected.
# None puts every assay back on a single left axis.
RAW_RIGHT_AXIS_ASSAY = 'MTT'
# Per-assay label for that axis; unlisted assays fall back to '<assay> raw metric'.
ASSAY_RAW_YLABEL = {
'Castor': 'Castor confluence (fraction)',
'MTT': 'MTT absorbance (a.u.)',
}
# =====================================================================
PLOT_DIR = os.path.join(DATA_DIR, 'plots')
# Stem for exported CSVs — the experiment folder name (DATA_DIR is often '.').
# If DATA_DIR is a generic 'Data' wrapper, name exports after the experiment
# folder above it (e.g. .../LS180/Data -> 'LS180') rather than after 'Data'.
_stem_dir = os.path.abspath(DATA_DIR)
if os.path.basename(_stem_dir).lower() == 'data':
_stem_dir = os.path.dirname(_stem_dir)
EXPORT_STEM = os.path.basename(_stem_dir)
os.makedirs(PLOT_DIR, exist_ok=True)
plot_counter = [0]
def save_fig(title=''):
plot_counter[0] += 1
fname = os.path.join(PLOT_DIR, f'plot_{plot_counter[0]:02d}.png')
plt.savefig(fname, dpi=120, bbox_inches='tight')
plt.close()
display(IPImage(filename=fname))
# Auto-discover data: readout CSVs in DATA_DIR ($ anchor keeps Windows
# ':Zone.Identifier' sidecar files out).
_fname_pat = re.compile(r'.*_P(?:late)?(\d+)_D(\d+)\.csv$', re.IGNORECASE)
csv_data_files = sorted([f for f in os.listdir(DATA_DIR) if _fname_pat.match(f)])
# Plate layout CSVs live in DATA_DIR; group 2 is the optional assay suffix.
_layout_pat = re.compile(r'Plate(\d+)_layout(?:_([A-Za-z0-9]+))?\.csv$', re.IGNORECASE)
PLATE_LAYOUTS = sorted([f for f in os.listdir(DATA_DIR) if _layout_pat.match(f)])
assert len(PLATE_LAYOUTS) >= 1, f'No Plate*_layout*.csv found in {DATA_DIR}'
print(f'Data dir : {DATA_DIR}')
print(f'Readouts : {len(csv_data_files)} data file(s)')
print(f'Layouts : {len(PLATE_LAYOUTS)} file(s) — {PLATE_LAYOUTS}')
print(f'X-axis : {SCALE_MIN_CONC} – {SCALE_MAX_CONC if SCALE_MAX_CONC is not None else "all retained concentrations"}')
print(f'Excl ABs : {SCALE_EXCL_ABS} (excluded from y-axis range only)')
print(f'Assays : {[lbl for _, lbl in ASSAY_PATTERNS]} (default {DEFAULT_ASSAY})')Data dir : /content/drive/MyDrive/20260827 Cytotox - Hu64_VPE6_GE6/Capan Combine Analysis-20260910/Data
Readouts : 4 data file(s)
Layouts : 4 file(s) — ['Plate1_layout_Castor.csv', 'Plate1_layout_MTT.csv', 'Plate2_layout_Castor.csv', 'Plate2_layout_MTT.csv']
X-axis : 0 – all retained concentrations
Excl ABs : [] (excluded from y-axis range only)
Assays : ['Castor', 'MTT'] (default Castor)
# ===========================================================================
# 1. Load readout data, tag each file with its assay, merge by assay
# Key is (Assay, Plate, Day). Plate numbers repeat across assays, so the
# assay is what separates Castor P1 from MTT P1; a repeated key is skipped,
# never silently averaged in.
# ===========================================================================
print('\n=== 1. Load readout data (merged by assay) ===')
csv_files = sorted([f for f in os.listdir(DATA_DIR) if f.lower().endswith('.csv')])
print(f'{len(csv_files)} CSV files found')
def assay_from_filename(fname):
"""Assay label implied by a readout filename, or None if no pattern matches."""
for pat, label in ASSAY_PATTERNS:
if re.search(pat, fname, re.IGNORECASE):
return label
return None
frames = []
seen = {}
for fname in csv_files:
m = _fname_pat.match(fname)
if not m or _layout_pat.match(fname):
continue
plate_num = int(m.group(1))
day = int(m.group(2))
assay = FILE_ASSAY.get(fname) or assay_from_filename(fname) or DEFAULT_ASSAY
key = (assay, plate_num, day)
if key in seen:
print(f' SKIP duplicate {assay} P{plate_num} D{day}: {fname} '
f'(already loaded {seen[key]})')
continue
seen[key] = fname
# utf-8-sig: the MTT exports carry a BOM on the first header cell.
tmp = pd.read_csv(os.path.join(DATA_DIR, fname), index_col=0, encoding='utf-8-sig')
tmp.index.name = 'Well'
tmp = tmp.reset_index()
tmp['Plate'] = plate_num
tmp['Day'] = day
tmp['Assay'] = assay
frames.append(tmp)
print(f' {assay:<8} P{plate_num} D{day} <- {fname}')
assert frames, f'No readout CSVs matched {_fname_pat.pattern} in {DATA_DIR}'
df_raw = pd.concat(frames, ignore_index=True)
df_raw = df_raw.drop(columns=['Total cell area'], errors='ignore')
ASSAYS = sorted(df_raw.Assay.unique())
ASSAY_PLATES = {a: sorted(df_raw.loc[df_raw.Assay == a, 'Plate'].unique().tolist())
for a in ASSAYS}
print(f'\nCombined raw data: {df_raw.shape[0]} rows x {df_raw.shape[1]} cols')
print(f'Assays: {ASSAYS}')
for _a in ASSAYS:
_sub = df_raw[df_raw.Assay == _a]
print(f' {_a:<8} plates {ASSAY_PLATES[_a]} '
f'days {sorted(_sub.Day.unique().tolist())} ({len(_sub)} wells)')
_shared = sorted(set(ASSAY_PLATES[ASSAYS[0]]).intersection(
*[set(ASSAY_PLATES[a]) for a in ASSAYS[1:]])) if len(ASSAYS) > 1 else []
if _shared:
print(f' NOTE: plate number(s) {_shared} are reused across assays — '
f'kept distinct by (Assay, Plate).')
=== 1. Load readout data (merged by assay) ===
12 CSV files found
Castor P1 D7 <- CASTOR-20260827_PCM_SH_Capan_63GE8_IsoGE8_P1_D7.csv
Castor P2 D7 <- CASTOR-PCM_SH_20260827_Capan_64GE6-64VPE6_P2_D7.csv
MTT P1 D7 <- MTT-PCM_SH_Capan_20260827_63GE8-IsoGE8_P1_D7.csv
MTT P2 D7 <- MTT-PCM_SH_Capan_20260827_64GE6-64VPE6_P2_D7.csv
Combined raw data: 240 rows x 5 cols
Assays: ['Castor', 'MTT']
Castor plates [1, 2] days [7] (120 wells)
MTT plates [1, 2] days [7] (120 wells)
NOTE: plate number(s) [1, 2] are reused across assays — kept distinct by (Assay, Plate).
# ===========================================================================
# 2. Load plate layouts and merge on (Assay, Plate, Well)
# Plate<n>_layout_<Assay>.csv binds a layout to one assay; an assay-less
# Plate<n>_layout.csv is broadcast to every assay in the data.
# ===========================================================================
print('\n=== 2. Load plate layouts and merge ===')
layout_frames = []
for lf in PLATE_LAYOUTS:
m = _layout_pat.match(lf)
plate_num = int(m.group(1))
layout_assay = m.group(2) # None -> applies to every assay
if layout_assay is None:
targets = list(ASSAYS)
else:
targets = [a for a in ASSAYS if a.lower() == layout_assay.lower()]
if not targets:
print(f' WARNING: {lf} names assay {layout_assay!r}, which no readout '
f'file uses ({ASSAYS}) — layout ignored')
continue
tmp = pd.read_csv(os.path.join(DATA_DIR, lf), encoding='utf-8-sig')
tmp.rename(columns={'Cell': 'Well'}, inplace=True)
tmp['Plate'] = plate_num
for a in targets:
layout_frames.append(tmp.assign(Assay=a))
print(f' P{plate_num} [{", ".join(targets)}] drugs: {sorted(tmp.Drug.unique())} '
f'<- {lf}')
assert layout_frames, 'No plate layout matched an assay present in the readout data'
layouts = pd.concat(layout_frames, ignore_index=True)
_dups = layouts.duplicated(subset=['Assay', 'Plate', 'Well']).sum()
assert _dups == 0, (f'{_dups} duplicate (Assay, Plate, Well) layout entries — two '
f'layout files claim the same assay/plate')
print(f'Layout entries: {len(layouts)} ({len(PLATE_LAYOUTS)} file(s), '
f'{layouts.groupby(["Assay", "Plate"]).ngroups} assay×plate combination(s))')
df = df_raw.merge(layouts[['Assay', 'Plate', 'Well', 'Drug', 'Conc']],
on=['Assay', 'Plate', 'Well'], how='left')
unmatched = df.Drug.isna().sum()
if unmatched > 0:
print(f'WARNING: {unmatched} wells did not match a layout entry')
display(df[df.Drug.isna()].groupby(['Assay', 'Plate']).size()
.rename('unmatched wells').to_frame())
else:
print('All wells matched their layout.')
print(f'Drugs: {sorted(df.Drug.dropna().unique().tolist())}')
print(f'Concentrations: {sorted(df.Conc.dropna().unique().tolist())}')
# Drug x Assay coverage — every populated cell here becomes its own curve.
display(Markdown('### Wells per Drug × Assay — each pair is a separate series'))
display(pd.crosstab(df.Drug, df.Assay))
# Auto-derive concentration parameters from the data
nonzero_concs = df.loc[df.Conc > 0, 'Conc']
CONC_MIN = nonzero_concs.min()
CONC_MAX = nonzero_concs.max()
PSEUDO_CONC = CONC_MIN / 10.0
df['Conc_orig'] = df['Conc'].copy()
df.loc[df.Conc == 0, 'Conc'] = PSEUDO_CONC
print(f'Conc range: {CONC_MIN:.4g} – {CONC_MAX:.4g} nM')
print(f'Conc==0 replaced with pseudo-conc {PSEUDO_CONC:.4g} nM')
=== 2. Load plate layouts and merge ===
P1 [Castor] drugs: ['63GE8', 'IsoGE8'] <- Plate1_layout_Castor.csv
P1 [MTT] drugs: ['63GE8', 'IsoGE8'] <- Plate1_layout_MTT.csv
P2 [Castor] drugs: ['64GE6', '64VPE6'] <- Plate2_layout_Castor.csv
P2 [MTT] drugs: ['64GE6', '64VPE6'] <- Plate2_layout_MTT.csv
Layout entries: 240 (4 file(s), 4 assay×plate combination(s))
All wells matched their layout.
Drugs: ['63GE8', '64GE6', '64VPE6', 'IsoGE8']
Concentrations: [0.0, 0.006, 0.026, 0.12, 0.54, 2.44, 11.0, 49.4, 222.2, 1000.0]
| Assay | Castor | MTT |
|---|---|---|
| Drug | ||
| 63GE8 | 30 | 30 |
| 64GE6 | 30 | 30 |
| 64VPE6 | 30 | 30 |
| IsoGE8 | 30 | 30 |
Conc range: 0.006 – 1000 nM
Conc==0 replaced with pseudo-conc 0.0006 nM
# ===========================================================================
# 3. Raw metric QC (no Day-0 normalization)
# The raw metric is used as measured. Day-0 confluence is NOT used as a
# per-well baseline; all dose-response normalization is done against the
# untreated wells of the same assay/plate/drug/day (%Viable, sec. 7b).
# NOTE the raw units differ between assays (Castor = confluence fraction,
# MTT = absorbance) — they are only comparable once normalized to %Viable.
# ===========================================================================
print('\n=== 3. Raw metric QC (no Day-0 normalization) ===')
if 0 in set(df.Day.unique()):
print('NOTE: a Day 0 scan is present but is NOT used as a baseline. '
'It is treated as an ordinary timepoint and excluded from curve fitting.')
else:
print('No Day 0 scan present — none required.')
display(Markdown('### Raw metric stats by Assay and Day:'))
display(df.groupby(['Assay', 'Day'])['Confluence'].describe().round(3))
display(Markdown('### Raw metric stats by Assay, Plate and Day:'))
display(df.groupby(['Assay', 'Plate', 'Day'])['Confluence'].describe().round(3))
# Drug-by-drug fragmentation filter, run independently within each assay.
# Compare the mean raw metric at each concentration with the immediately
# preceding concentration for that assay/drug/day.
_filter_means = (
df.groupby(['Assay', 'Drug', 'Day', 'Conc'], dropna=False)[REVERSAL_FILTER_METRIC]
.mean().rename('Reversal_Filter_Mean').reset_index()
.sort_values(['Assay', 'Drug', 'Day', 'Conc'])
)
_filter_means['Reversal_Previous_Mean'] = (
_filter_means.groupby(['Assay', 'Drug', 'Day'])['Reversal_Filter_Mean'].shift(1)
)
_prev = _filter_means['Reversal_Previous_Mean']
_curr = _filter_means['Reversal_Filter_Mean']
_filter_means['Reversal_Pct_Increase'] = np.where(
_prev > 0, (_curr - _prev) / _prev,
np.where((_prev == 0) & (_curr > 0), np.inf, np.nan)
)
_filter_means['Exclude_25pct_Reversal'] = (
_filter_means['Reversal_Pct_Increase'] > REVERSAL_INCREASE_THRESHOLD
)
df = df.drop(columns=['Reversal_Filter_Mean', 'Reversal_Previous_Mean',
'Reversal_Pct_Increase', 'Exclude_25pct_Reversal'],
errors='ignore')
df = df.merge(
_filter_means[['Assay', 'Drug', 'Day', 'Conc', 'Reversal_Filter_Mean',
'Reversal_Previous_Mean', 'Reversal_Pct_Increase',
'Exclude_25pct_Reversal']],
on=['Assay', 'Drug', 'Day', 'Conc'], how='left'
)
df['Exclude_25pct_Reversal'] = df['Exclude_25pct_Reversal'].fillna(False).astype(bool)
excluded_reversals = _filter_means[_filter_means.Exclude_25pct_Reversal].copy()
print(f'\n25% reversal filter metric: {REVERSAL_FILTER_METRIC} (applied within each assay)')
print(f'Excluded assay/drug/day/concentration groups: {len(excluded_reversals)}')
if not excluded_reversals.empty:
display(excluded_reversals[['Assay', 'Drug', 'Day', 'Conc', 'Reversal_Previous_Mean',
'Reversal_Filter_Mean', 'Reversal_Pct_Increase']]
.round(3).style.hide(axis='index'))
=== 3. Raw metric QC (no Day-0 normalization) ===
No Day 0 scan present — none required.
| count | mean | std | min | 25% | 50% | 75% | max | ||
|---|---|---|---|---|---|---|---|---|---|
| Assay | Day | ||||||||
| Castor | 7 | 120.0 | 0.082 | 0.038 | 0.031 | 0.057 | 0.077 | 0.103 | 0.338 |
| MTT | 7 | 120.0 | 0.121 | 0.041 | 0.060 | 0.085 | 0.116 | 0.153 | 0.206 |
| count | mean | std | min | 25% | 50% | 75% | max | |||
|---|---|---|---|---|---|---|---|---|---|---|
| Assay | Plate | Day | ||||||||
| Castor | 1 | 7 | 60.0 | 0.090 | 0.042 | 0.038 | 0.064 | 0.087 | 0.101 | 0.338 |
| 2 | 7 | 60.0 | 0.074 | 0.033 | 0.031 | 0.046 | 0.064 | 0.106 | 0.147 | |
| MTT | 1 | 7 | 60.0 | 0.127 | 0.040 | 0.061 | 0.099 | 0.129 | 0.157 | 0.206 |
| 2 | 7 | 60.0 | 0.115 | 0.041 | 0.060 | 0.077 | 0.107 | 0.149 | 0.204 |
25% reversal filter metric: Confluence (applied within each assay)
Excluded assay/drug/day/concentration groups: 4
| Assay | Drug | Day | Conc | Reversal_Previous_Mean | Reversal_Filter_Mean | Reversal_Pct_Increase |
|---|---|---|---|---|---|---|
| Castor | 63GE8 | 7 | 222.200000 | 0.053000 | 0.080000 | 0.494000 |
| Castor | 63GE8 | 7 | 1000.000000 | 0.080000 | 0.200000 | 1.497000 |
| Castor | IsoGE8 | 7 | 0.006000 | 0.081000 | 0.107000 | 0.323000 |
| Castor | IsoGE8 | 7 | 1000.000000 | 0.055000 | 0.076000 | 0.387000 |
# ===========================================================================
# 3b. Replicate variance & outlier well detection — per Drug × Concentration
# Plate layout: replicates run DOWN a column. The three wells of one
# triplicate share a column and differ only by plate ROW (e.g. E2/F2/G2),
# while concentration varies ACROSS columns (col 2 = top dose ... col 11 =
# untreated). So one replicate group is (Assay, Plate, Drug, Day, Conc) and
# its members are distinguished by row.
#
# The replicate axis is detected from the data rather than assumed, so this
# still works if a plate is ever laid out the other way round.
#
# Run on the RAW metric, before any normalization: Castor confluence and MTT
# absorbance are assessed separately and never compared, since their scales
# and their noise floors are completely different.
# ===========================================================================
print('\n=== 3b. Replicate variance & outlier wells (raw metric, per assay) ===')
# --- USER INPUT: what counts as a suspect replicate ------------------------
REPLICATE_CV_THRESHOLD = 25.0 # % CV above which a triplicate is flagged
REPLICATE_ROW_BIAS_PCT = 15.0 # mean % deviation above which a whole replicate
# position (plate row) is called systematically off
# ---------------------------------------------------------------------------
_REP_KEY = ['Assay', 'Plate', 'Drug', 'Day', 'Conc']
_rep_metric = 'Confluence' # the raw column; units differ per assay
_well_pat = re.compile(r'^\s*([A-Za-z]+)\s*(\d+)\s*$')
def _well_row(w):
m = _well_pat.match(str(w))
return m.group(1).upper() if m else np.nan
def _well_col(w):
m = _well_pat.match(str(w))
return int(m.group(2)) if m else np.nan
def _cv_pct(values):
"""Coefficient of variation (%) — scale-free, so Castor and MTT are comparable."""
v = np.asarray(values, dtype=float)
v = v[np.isfinite(v)]
if len(v) < 2:
return np.nan
m = v.mean()
if m == 0:
return np.nan
return 100.0 * v.std(ddof=1) / m
_rep = df.dropna(subset=[_rep_metric]).copy()
_rep['Row'] = _rep.Well.map(_well_row)
_rep['Col'] = _rep.Well.map(_well_col)
# Deviation of each well from the median of its OWN triplicate. The median is
# used rather than the mean because with n = 3 a single bad well drags the mean
# toward itself and masks its own deviation.
_grp_median = _rep.groupby(_REP_KEY, dropna=False)[_rep_metric].transform('median')
_rep['Rep_Median'] = _grp_median
_rep['Dev_pct'] = np.where(_grp_median != 0,
100.0 * (_rep[_rep_metric] - _grp_median) / _grp_median,
np.nan)
# Which plate axis separates the members of a replicate group?
_axis_var = _rep.groupby(_REP_KEY, dropna=False)[['Row', 'Col']].nunique()
_rep_axis = ('Row' if _axis_var['Row'].mean() >= _axis_var['Col'].mean() else 'Col')
_other_axis = 'Col' if _rep_axis == 'Row' else 'Row'
print(f' Replicate axis detected: members of a triplicate differ by {_rep_axis} '
f'(constant {_other_axis}); {_other_axis} carries concentration.')
# --- Per-group statistics ---------------------------------------------------
_rep_rows = []
for _key, _g in _rep.groupby(_REP_KEY, dropna=False):
_v = _g[_rep_metric].to_numpy(dtype=float)
_w = _g.Well.to_numpy()
_pos = _g[_rep_axis].to_numpy()
_med = float(np.median(_v))
_cv = _cv_pct(_v)
# The well furthest from the triplicate median is the candidate culprit.
_i = int(np.argmax(np.abs(_v - _med)))
_keep = [j for j in range(len(_v)) if j != _i]
_cv_drop = _cv_pct(_v[_keep]) if len(_keep) >= 2 else np.nan
# Three-way verdict. "one suspect well" means the remaining replicates agree
# once that well is removed — i.e. a single well is to blame. "scattered"
# means the whole group disagrees and no single well explains it.
if not np.isfinite(_cv) or _cv <= REPLICATE_CV_THRESHOLD:
_verdict, _suspect = 'OK', ''
elif np.isfinite(_cv_drop) and _cv_drop <= REPLICATE_CV_THRESHOLD:
_verdict, _suspect = 'one suspect well', _w[_i]
else:
_verdict, _suspect = 'scattered', ''
_others = ' / '.join(f'{x:.4g}' for x in _v[_keep])
# Plate-layout coordinates, so the table can be read against the physical
# plate: each drug occupies a contiguous block of replicate positions
# (rows B-D, E-G, ...) and each group sits in one concentration position
# (column). Sorting on these reproduces plate reading order.
_pos_sorted = sorted(str(p) for p in set(_pos))
_conc_pos = _g[_other_axis].dropna()
_conc_pos = _conc_pos.iloc[0] if len(_conc_pos) else np.nan
_rep_rows.append(dict(zip(_REP_KEY, _key)) | {
'Wells': '/'.join(sorted(map(str, _w))),
'Block': _pos_sorted[0] if _pos_sorted else '',
'Conc_Pos': _conc_pos,
'N': len(_v), 'Mean': _v.mean(), 'SD': _v.std(ddof=1) if len(_v) > 1 else np.nan,
'CV_pct': _cv, 'Min': _v.min(), 'Max': _v.max(),
'Worst_Well': _w[_i], 'Worst_Pos': _pos[_i],
'Worst_Value': _v[_i], 'Worst_Dev_pct': 100.0 * (_v[_i] - _med) / _med if _med else np.nan,
'Other_Reps': _others, 'CV_pct_drop1': _cv_drop,
'Verdict': _verdict, 'Suspect_Well': _suspect,
'Untreated': bool((_g.Conc_orig == 0).all()),
'Reversal_Excluded': bool(_g.Exclude_25pct_Reversal.any()),
})
# Canonical order = plate reading order: assay, plate, drug's replicate block
# (top block first), then across the concentration axis (column 2 -> 11, i.e.
# top dose -> untreated). Views that need worst-first re-sort explicitly.
df_replicate_qc = pd.DataFrame(_rep_rows).sort_values(
['Assay', 'Plate', 'Block', 'Conc_Pos']).reset_index(drop=True)
# --- Systematic bias along the replicate axis -------------------------------
# Only the replicate axis is interpretable here: the other axis IS concentration,
# so a "bias" along it would just be the dose-response curve.
_pos_bias = (_rep.groupby(['Assay', 'Plate', _rep_axis])['Dev_pct']
.mean().rename('Mean_Dev_pct').reset_index())
_pos_bias['Systematic'] = _pos_bias.Mean_Dev_pct.abs() >= REPLICATE_ROW_BIAS_PCT
_bias_lookup = {(a, p, r): m for a, p, r, m, _s in _pos_bias.itertuples(index=False)}
def _likely_cause(row):
"""Distinguish a one-off bad well from a whole replicate position being off."""
if not row.Suspect_Well:
return ''
_b = _bias_lookup.get((row.Assay, row.Plate, row.Worst_Pos), np.nan)
if np.isfinite(_b) and abs(_b) >= REPLICATE_ROW_BIAS_PCT:
return f'{_rep_axis} {row.Worst_Pos} biased ({_b:+.0f}% plate-wide)'
return 'isolated well'
df_replicate_qc['Likely_Cause'] = df_replicate_qc.apply(_likely_cause, axis=1)
_sig4 = lambda x: f'{x:.4g}'
# Plate banding: alternating black / grey down the Plate column, so consecutive
# plate blocks are separable at a glance when scanning the table.
# Keyed on the plate number itself (odd = black, even = grey) rather than on row
# position, so a given plate keeps the same shade in every table.
# NB: Plate is kept as a DATA column rather than an index level — pandas
# sparsifies repeated MultiIndex labels and `Styler.map_index` then emits CSS
# for only the first row of each run, so index-level banding does not render.
_PLATE_BAND = {0: 'background-color: #757575; color: #ffffff;', # even plates: grey
1: 'background-color: #1f1f1f; color: #ffffff;'} # odd plates: black
def _plate_band(v):
try:
return _PLATE_BAND[int(v) % 2]
except (TypeError, ValueError):
return ''
# --- Table 1: per-assay overview -------------------------------------------
_ov = (df_replicate_qc.groupby('Assay')
.agg(Groups=('CV_pct', 'size'),
Median_CV=('CV_pct', 'median'),
P90_CV=('CV_pct', lambda s: s.quantile(0.90)),
Max_CV=('CV_pct', 'max'),
Flagged=('Verdict', lambda s: int((s != 'OK').sum())),
Suspect_Well=('Verdict', lambda s: int((s == 'one suspect well').sum())),
Scattered=('Verdict', lambda s: int((s == 'scattered').sum()))))
_ov.insert(0, 'Raw metric', [ASSAY_RAW_YLABEL.get(a, 'raw metric') for a in _ov.index])
display(Markdown('### Replicate variance overview — per assay'))
display(Markdown(
f'Replicate group = one triplicate: `{"`, `".join(_REP_KEY)}`. '
f'A group is flagged when its CV exceeds **{REPLICATE_CV_THRESHOLD:.0f}%**. '
'Castor and MTT are summarised separately — their raw units and noise floors '
'are not comparable.'))
display(_ov.style.format({'Median_CV': '{:.1f}', 'P90_CV': '{:.1f}',
'Max_CV': '{:.1f}'}, na_rep='—'))
# --- Table 2: full per-replicate variance table, in plate order -------------
display(Markdown(f'### Replicate variance by Drug × Concentration (plate layout order)'))
display(Markdown(
f'Rows follow the physical plate: assay, plate, each drug\'s '
f'{_rep_axis.lower()} block top-first, then across the '
f'{_other_axis.lower()}s (top dose → untreated) — so this table can be read '
'side by side with the plate map. `Wells` lists the triplicate. '
'`Worst_Well` is the well furthest from its triplicate median and '
'`CV_pct_drop1` is what the CV would become without it; when dropping it '
'brings the group under threshold, a single well is to blame.'))
_tbl2 = df_replicate_qc.set_index(['Assay', 'Drug', 'Day', 'Conc'])
display(
_tbl2[['Plate', 'Wells', 'N', 'Mean', 'SD', 'CV_pct', 'Worst_Well', 'Worst_Value',
'Worst_Dev_pct', 'Other_Reps', 'CV_pct_drop1', 'Verdict', 'Untreated']].style
.format({'Mean': _sig4, 'SD': _sig4, 'CV_pct': '{:.1f}',
'Worst_Value': _sig4, 'Worst_Dev_pct': '{:+.1f}',
'CV_pct_drop1': '{:.1f}'}, na_rep='—')
.format_index(lambda v: f'{v:g}', axis=0, level='Conc')
.map(_plate_band, subset=['Plate'])
.map(lambda v: 'font-weight: bold' if v in ('one suspect well', 'scattered') else None,
subset=['Verdict'])
)
# --- Table 3: the actionable list ------------------------------------------
# Grouped by drug, inheriting the plate-layout order of the table above, so a
# drug's flagged concentrations read together and in dose order rather than
# being split apart by severity.
_flagged = df_replicate_qc[df_replicate_qc.Verdict != 'OK'].copy()
display(Markdown('### Suspect wells — groups above the CV threshold (grouped by drug)'))
if _flagged.empty:
display(Markdown(f'_No replicate group exceeds {REPLICATE_CV_THRESHOLD:.0f}% CV._'))
else:
_t3 = _flagged.set_index(['Assay', 'Drug', 'Conc'])
display(
_t3[['Plate', 'CV_pct', 'Worst_Well', 'Worst_Value', 'Other_Reps',
'Worst_Dev_pct', 'CV_pct_drop1', 'Verdict', 'Likely_Cause', 'Untreated',
'Reversal_Excluded']].style
.format({'CV_pct': '{:.1f}', 'Worst_Value': _sig4,
'Worst_Dev_pct': '{:+.1f}', 'CV_pct_drop1': '{:.1f}'}, na_rep='—')
.format_index(lambda v: f'{v:g}', axis=0, level='Conc')
.map(_plate_band, subset=['Plate'])
)
# --- Table 4: is a whole replicate position off? ----------------------------
display(Markdown(f'### Systematic bias by replicate position (plate {_rep_axis.lower()})'))
display(Markdown(
f'Mean % deviation of each plate {_rep_axis.lower()} from its own triplicate '
f'median. Near 0 = that replicate position tracks the others. '
f'Beyond ±{REPLICATE_ROW_BIAS_PCT:.0f}% the position is systematically off, '
'and individual wells flagged there are symptoms of that, not independent '
'bad wells. The other plate axis is not shown: it carries concentration, so '
'a "bias" along it would just be the dose response.'))
_bias_pivot = _pos_bias.pivot_table(index=['Assay', 'Plate'], columns=_rep_axis,
values='Mean_Dev_pct')
_bias_pivot = _bias_pivot.rename_axis(columns=None)
display(
_bias_pivot.style
.format('{:+.1f}', na_rep='—')
.map(lambda v: 'font-weight: bold' if (pd.notna(v) and abs(v) >= REPLICATE_ROW_BIAS_PCT)
else None)
)
# --- Printed summary --------------------------------------------------------
print(f'\n Replicate groups: {len(df_replicate_qc)} '
f'({df_replicate_qc.N.min()}–{df_replicate_qc.N.max()} wells each)')
print(f' Above {REPLICATE_CV_THRESHOLD:.0f}% CV: {int((df_replicate_qc.Verdict != "OK").sum())}'
f' [{int((df_replicate_qc.Verdict == "one suspect well").sum())} with a single '
f'suspect well, {int((df_replicate_qc.Verdict == "scattered").sum())} scattered]')
_sys = _pos_bias[_pos_bias.Systematic]
if not _sys.empty:
print(f'\n Systematically biased replicate positions '
f'(|mean deviation| >= {REPLICATE_ROW_BIAS_PCT:.0f}%):')
for _r in _sys.itertuples(index=False):
print(f' {_r.Assay} plate {_r.Plate}, {_rep_axis.lower()} '
f'{getattr(_r, _rep_axis)}: {_r.Mean_Dev_pct:+.1f}%')
print(' -> re-check seeding/edge effects for these positions before '
'blaming individual wells.')
_untx_bad = df_replicate_qc[(df_replicate_qc.Verdict != 'OK') & df_replicate_qc.Untreated]
if not _untx_bad.empty:
print(f'\n WARNING: {len(_untx_bad)} flagged group(s) are UNTREATED wells, which set '
'the 100% baseline —')
print(' an outlier there shifts %Viable for every concentration of that drug:')
for _r in _untx_bad.itertuples(index=False):
print(f' {_r.Assay} plate {_r.Plate} {_r.Drug}: CV {_r.CV_pct:.1f}%, '
f'worst well {_r.Worst_Well} ({_r.Worst_Dev_pct:+.0f}%)'
+ (f' — {_r.Likely_Cause}' if _r.Likely_Cause else ''))
=== 3b. Replicate variance & outlier wells (raw metric, per assay) ===
Replicate axis detected: members of a triplicate differ by Row (constant Col); Col carries concentration.
Replicate group = one triplicate: Assay, Plate, Drug, Day, Conc. A group is flagged when its CV exceeds 25%. Castor and MTT are summarised separately — their raw units and noise floors are not comparable.
| Raw metric | Groups | Median_CV | P90_CV | Max_CV | Flagged | Suspect_Well | Scattered | |
|---|---|---|---|---|---|---|---|---|
| Assay | ||||||||
| Castor | Castor confluence (fraction) | 40 | 11.8 | 22.7 | 64.9 | 3 | 1 | 2 |
| MTT | MTT absorbance (a.u.) | 40 | 8.7 | 16.6 | 19.9 | 0 | 0 | 0 |
Rows follow the physical plate: assay, plate, each drug’s row block top-first, then across the cols (top dose → untreated) — so this table can be read side by side with the plate map. Wells lists the triplicate. Worst_Well is the well furthest from its triplicate median and CV_pct_drop1 is what the CV would become without it; when dropping it brings the group under threshold, a single well is to blame.
| Plate | Wells | N | Mean | SD | CV_pct | Worst_Well | Worst_Value | Worst_Dev_pct | Other_Reps | CV_pct_drop1 | Verdict | Untreated | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Assay | Drug | Day | Conc | |||||||||||||
| Castor | 63GE8 | 7 | 1000 | 1 | B2/C2/D2 | 3 | 0.1996 | 0.127 | 63.6 | D2 | 0.338 | +96.4 | 0.0886 / 0.1721 | 45.3 | scattered | False |
| 222.2 | 1 | B3/C3/D3 | 3 | 0.07992 | 0.0519 | 64.9 | D3 | 0.1383 | +121.4 | 0.039 / 0.06247 | 32.7 | scattered | False | |||
| 49.4 | 1 | B4/C4/D4 | 3 | 0.0535 | 0.01403 | 26.2 | B4 | 0.03774 | -35.0 | 0.06464 / 0.05811 | 7.5 | one suspect well | False | |||
| 11 | 1 | B5/C5/D5 | 3 | 0.0604 | 0.007224 | 12.0 | B5 | 0.05221 | -17.3 | 0.06586 / 0.06314 | 3.0 | OK | False | |||
| 2.44 | 1 | B6/C6/D6 | 3 | 0.06872 | 0.01508 | 21.9 | B6 | 0.05142 | -32.0 | 0.07565 / 0.0791 | 3.2 | OK | False | |||
| 0.54 | 1 | B7/C7/D7 | 3 | 0.06179 | 0.003831 | 6.2 | B7 | 0.05738 | -9.9 | 0.06367 / 0.06431 | 0.7 | OK | False | |||
| 0.12 | 1 | B8/C8/D8 | 3 | 0.07039 | 0.009447 | 13.4 | B8 | 0.05992 | -17.9 | 0.07296 / 0.07828 | 5.0 | OK | False | |||
| 0.026 | 1 | B9/C9/D9 | 3 | 0.08111 | 0.0153 | 18.9 | B9 | 0.06541 | -20.2 | 0.09597 / 0.08196 | 11.1 | OK | False | |||
| 0.006 | 1 | B10/C10/D10 | 3 | 0.1026 | 0.01251 | 12.2 | D10 | 0.1164 | +17.0 | 0.09197 / 0.09953 | 5.6 | OK | False | |||
| 0.0006 | 1 | B11/C11/D11 | 3 | 0.08701 | 0.007421 | 8.5 | C11 | 0.07865 | -12.2 | 0.08954 / 0.09283 | 2.5 | OK | True | |||
| IsoGE8 | 7 | 1000 | 1 | E2/F2/G2 | 3 | 0.07569 | 0.01229 | 16.2 | F2 | 0.08907 | +21.9 | 0.07308 / 0.06491 | 8.4 | OK | False | |
| 222.2 | 1 | E3/F3/G3 | 3 | 0.05456 | 0.01023 | 18.8 | E3 | 0.04287 | -27.3 | 0.05896 / 0.06186 | 3.4 | OK | False | |||
| 49.4 | 1 | E4/F4/G4 | 3 | 0.06804 | 0.009257 | 13.6 | F4 | 0.05791 | -17.5 | 0.07605 / 0.07016 | 5.7 | OK | False | |||
| 11 | 1 | E5/F5/G5 | 3 | 0.09628 | 0.005802 | 6.0 | F5 | 0.08972 | -8.8 | 0.09838 / 0.1007 | 1.7 | OK | False | |||
| 2.44 | 1 | E6/F6/G6 | 3 | 0.1193 | 0.02699 | 22.6 | F6 | 0.09204 | -23.2 | 0.146 / 0.1198 | 13.9 | OK | False | |||
| 0.54 | 1 | E7/F7/G7 | 3 | 0.1007 | 0.003161 | 3.1 | G7 | 0.09705 | -4.9 | 0.1029 / 0.102 | 0.6 | OK | False | |||
| 0.12 | 1 | E8/F8/G8 | 3 | 0.1167 | 0.01359 | 11.6 | G8 | 0.1309 | +13.3 | 0.1155 / 0.1038 | 7.5 | OK | False | |||
| 0.026 | 1 | E9/F9/G9 | 3 | 0.1103 | 0.008853 | 8.0 | G9 | 0.1001 | -12.8 | 0.1149 / 0.1159 | 0.7 | OK | False | |||
| 0.006 | 1 | E10/F10/G10 | 3 | 0.1069 | 0.012 | 11.2 | F10 | 0.0932 | -16.8 | 0.112 / 0.1155 | 2.1 | OK | False | |||
| 0.0006 | 1 | E11/F11/G11 | 3 | 0.08084 | 0.01884 | 23.3 | E11 | 0.06041 | -28.6 | 0.08458 / 0.09753 | 10.1 | OK | True | |||
| 64GE6 | 7 | 1000 | 2 | B2/C2/D2 | 3 | 0.04692 | 0.001962 | 4.2 | D2 | 0.04907 | +5.6 | 0.04522 / 0.04647 | 1.9 | OK | False | |
| 222.2 | 2 | B3/C3/D3 | 3 | 0.0426 | 0.00829 | 19.5 | D3 | 0.03381 | -22.6 | 0.05028 / 0.0437 | 9.9 | OK | False | |||
| 49.4 | 2 | B4/C4/D4 | 3 | 0.04704 | 0.008324 | 17.7 | D4 | 0.0555 | +18.7 | 0.04677 / 0.03886 | 13.1 | OK | False | |||
| 11 | 2 | B5/C5/D5 | 3 | 0.05709 | 0.007112 | 12.5 | C5 | 0.06463 | +15.2 | 0.05051 / 0.05612 | 7.4 | OK | False | |||
| 2.44 | 2 | B6/C6/D6 | 3 | 0.05747 | 0.007416 | 12.9 | B6 | 0.04939 | -16.4 | 0.06396 / 0.05907 | 5.6 | OK | False | |||
| 0.54 | 2 | B7/C7/D7 | 3 | 0.06502 | 0.006575 | 10.1 | B7 | 0.05767 | -14.0 | 0.07035 / 0.06702 | 3.4 | OK | False | |||
| 0.12 | 2 | B8/C8/D8 | 3 | 0.07757 | 0.008874 | 11.4 | B8 | 0.06773 | -15.4 | 0.08496 / 0.08001 | 4.2 | OK | False | |||
| 0.026 | 2 | B9/C9/D9 | 3 | 0.1086 | 0.005476 | 5.0 | D9 | 0.1149 | +8.1 | 0.1047 / 0.1062 | 1.0 | OK | False | |||
| 0.006 | 2 | B10/C10/D10 | 3 | 0.107 | 0.005086 | 4.8 | D10 | 0.1011 | -7.8 | 0.1102 / 0.1097 | 0.3 | OK | False | |||
| 0.0006 | 2 | B11/C11/D11 | 3 | 0.1254 | 0.02004 | 16.0 | C11 | 0.147 | +20.6 | 0.1219 / 0.1074 | 8.9 | OK | True | |||
| 64VPE6 | 7 | 1000 | 2 | E2/F2/G2 | 3 | 0.04021 | 0.003385 | 8.4 | G2 | 0.04402 | +12.8 | 0.03757 / 0.03902 | 2.7 | OK | False | |
| 222.2 | 2 | E3/F3/G3 | 3 | 0.04015 | 0.001939 | 4.8 | G3 | 0.03792 | -8.1 | 0.04129 / 0.04126 | 0.0 | OK | False | |||
| 49.4 | 2 | E4/F4/G4 | 3 | 0.04266 | 0.003499 | 8.2 | F4 | 0.03884 | -10.6 | 0.0457 / 0.04345 | 3.6 | OK | False | |||
| 11 | 2 | E5/F5/G5 | 3 | 0.03579 | 0.003929 | 11.0 | F5 | 0.03126 | -17.5 | 0.03822 / 0.03789 | 0.6 | OK | False | |||
| 2.44 | 2 | E6/F6/G6 | 3 | 0.05625 | 0.005301 | 9.4 | G6 | 0.05018 | -14.4 | 0.05995 / 0.05863 | 1.6 | OK | False | |||
| 0.54 | 2 | E7/F7/G7 | 3 | 0.08088 | 0.009098 | 11.2 | G7 | 0.07039 | -17.9 | 0.0857 / 0.08656 | 0.7 | OK | False | |||
| 0.12 | 2 | E8/F8/G8 | 3 | 0.1046 | 0.01442 | 13.8 | E8 | 0.08912 | -16.7 | 0.1176 / 0.107 | 6.7 | OK | False | |||
| 0.026 | 2 | E9/F9/G9 | 3 | 0.116 | 0.02198 | 19.0 | F9 | 0.09303 | -21.2 | 0.1369 / 0.118 | 10.4 | OK | False | |||
| 0.006 | 2 | E10/F10/G10 | 3 | 0.1207 | 0.006494 | 5.4 | F10 | 0.1132 | -8.9 | 0.1246 / 0.1243 | 0.2 | OK | False | |||
| 0.0006 | 2 | E11/F11/G11 | 3 | 0.1136 | 0.01014 | 8.9 | F11 | 0.102 | -14.2 | 0.1201 / 0.1189 | 0.7 | OK | True | |||
| MTT | 63GE8 | 7 | 1000 | 1 | B2/C2/D2 | 3 | 0.083 | 0.01004 | 12.1 | B2 | 0.0944 | +19.3 | 0.0755 / 0.0791 | 3.3 | OK | False |
| 222.2 | 1 | B3/C3/D3 | 3 | 0.08533 | 0.0141 | 16.5 | B3 | 0.1013 | +26.5 | 0.0801 / 0.0746 | 5.0 | OK | False | |||
| 49.4 | 1 | B4/C4/D4 | 3 | 0.09197 | 0.005095 | 5.5 | D4 | 0.0864 | -7.2 | 0.0964 / 0.0931 | 2.5 | OK | False | |||
| 11 | 1 | B5/C5/D5 | 3 | 0.1013 | 0.003041 | 3.0 | B5 | 0.1048 | +5.0 | 0.0998 / 0.0993 | 0.4 | OK | False | |||
| 2.44 | 1 | B6/C6/D6 | 3 | 0.1036 | 0.009886 | 9.5 | D6 | 0.0933 | -10.8 | 0.1046 / 0.113 | 5.5 | OK | False | |||
| 0.54 | 1 | B7/C7/D7 | 3 | 0.1224 | 0.01632 | 13.3 | D7 | 0.1046 | -16.9 | 0.1258 / 0.1367 | 5.9 | OK | False | |||
| 0.12 | 1 | B8/C8/D8 | 3 | 0.1342 | 0.02569 | 19.1 | D8 | 0.1067 | -22.8 | 0.1382 / 0.1576 | 9.3 | OK | False | |||
| 0.026 | 1 | B9/C9/D9 | 3 | 0.1666 | 0.006151 | 3.7 | D9 | 0.1595 | -6.2 | 0.17 / 0.1703 | 0.1 | OK | False | |||
| 0.006 | 1 | B10/C10/D10 | 3 | 0.1845 | 0.02458 | 13.3 | D10 | 0.1579 | -16.5 | 0.1891 / 0.2064 | 6.2 | OK | False | |||
| 0.0006 | 1 | B11/C11/D11 | 3 | 0.1936 | 0.009802 | 5.1 | B11 | 0.1837 | -5.2 | 0.1938 / 0.2033 | 3.4 | OK | True | |||
| IsoGE8 | 7 | 1000 | 1 | E2/F2/G2 | 3 | 0.0669 | 0.005462 | 8.2 | G2 | 0.0606 | -13.2 | 0.0698 / 0.0703 | 0.5 | OK | False | |
| 222.2 | 1 | E3/F3/G3 | 3 | 0.06837 | 0.00206 | 3.0 | F3 | 0.0707 | +4.6 | 0.0668 / 0.0676 | 0.8 | OK | False | |||
| 49.4 | 1 | E4/F4/G4 | 3 | 0.1028 | 0.004543 | 4.4 | E4 | 0.108 | +7.1 | 0.1008 / 0.0996 | 0.8 | OK | False | |||
| 11 | 1 | E5/F5/G5 | 3 | 0.1313 | 0.02265 | 17.3 | F5 | 0.1573 | +30.6 | 0.1161 / 0.1204 | 2.6 | OK | False | |||
| 2.44 | 1 | E6/F6/G6 | 3 | 0.1343 | 0.01157 | 8.6 | E6 | 0.1214 | -12.0 | 0.1379 / 0.1437 | 2.9 | OK | False | |||
| 0.54 | 1 | E7/F7/G7 | 3 | 0.1357 | 0.002816 | 2.1 | F7 | 0.1388 | +2.8 | 0.135 / 0.1333 | 0.9 | OK | False | |||
| 0.12 | 1 | E8/F8/G8 | 3 | 0.1452 | 0.006892 | 4.7 | G8 | 0.1376 | -6.5 | 0.151 / 0.1471 | 1.9 | OK | False | |||
| 0.026 | 1 | E9/F9/G9 | 3 | 0.1557 | 0.01996 | 12.8 | F9 | 0.1785 | +21.3 | 0.1414 / 0.1472 | 2.8 | OK | False | |||
| 0.006 | 1 | E10/F10/G10 | 3 | 0.1593 | 0.03162 | 19.9 | F10 | 0.194 | +27.8 | 0.1518 / 0.1321 | 9.8 | OK | False | |||
| 0.0006 | 1 | E11/F11/G11 | 3 | 0.1763 | 0.02261 | 12.8 | F11 | 0.2022 | +21.9 | 0.1607 / 0.1659 | 2.3 | OK | True | |||
| 64GE6 | 7 | 1000 | 2 | B2/C2/D2 | 3 | 0.0707 | 0.00652 | 9.2 | C2 | 0.0637 | -11.3 | 0.0766 / 0.0718 | 4.6 | OK | False | |
| 222.2 | 2 | B3/C3/D3 | 3 | 0.08053 | 0.008179 | 10.2 | D3 | 0.0899 | +16.9 | 0.0769 / 0.0748 | 2.0 | OK | False | |||
| 49.4 | 2 | B4/C4/D4 | 3 | 0.0828 | 0.006393 | 7.7 | C4 | 0.0755 | -11.7 | 0.0855 / 0.0874 | 1.6 | OK | False | |||
| 11 | 2 | B5/C5/D5 | 3 | 0.09037 | 0.01157 | 12.8 | C5 | 0.0771 | -19.4 | 0.0956 / 0.0984 | 2.0 | OK | False | |||
| 2.44 | 2 | B6/C6/D6 | 3 | 0.09617 | 0.007504 | 7.8 | C6 | 0.0878 | -10.8 | 0.1023 / 0.0984 | 2.7 | OK | False | |||
| 0.54 | 2 | B7/C7/D7 | 3 | 0.1123 | 0.002205 | 2.0 | C7 | 0.1099 | -2.7 | 0.1142 / 0.1129 | 0.8 | OK | False | |||
| 0.12 | 2 | B8/C8/D8 | 3 | 0.1347 | 0.006412 | 4.8 | C8 | 0.1421 | +8.3 | 0.1308 / 0.1312 | 0.2 | OK | False | |||
| 0.026 | 2 | B9/C9/D9 | 3 | 0.1598 | 0.004291 | 2.7 | D9 | 0.1549 | -4.3 | 0.1619 / 0.1627 | 0.3 | OK | False | |||
| 0.006 | 2 | B10/C10/D10 | 3 | 0.1773 | 0.01467 | 8.3 | D10 | 0.1615 | -10.2 | 0.1798 / 0.1905 | 4.1 | OK | False | |||
| 0.0006 | 2 | B11/C11/D11 | 3 | 0.1728 | 0.02343 | 13.6 | C11 | 0.1481 | -15.7 | 0.1756 / 0.1947 | 7.3 | OK | True | |||
| 64VPE6 | 7 | 1000 | 2 | E2/F2/G2 | 3 | 0.0666 | 0.005011 | 7.5 | F2 | 0.072 | +9.6 | 0.0657 / 0.0621 | 4.0 | OK | False | |
| 222.2 | 2 | E3/F3/G3 | 3 | 0.0653 | 0.007134 | 10.9 | F3 | 0.0733 | +16.3 | 0.063 / 0.0596 | 3.9 | OK | False | |||
| 49.4 | 2 | E4/F4/G4 | 3 | 0.07283 | 0.006341 | 8.7 | F4 | 0.0798 | +11.9 | 0.0713 / 0.0674 | 4.0 | OK | False | |||
| 11 | 2 | E5/F5/G5 | 3 | 0.07137 | 0.007778 | 10.9 | F5 | 0.0795 | +12.6 | 0.0706 / 0.064 | 6.9 | OK | False | |||
| 2.44 | 2 | E6/F6/G6 | 3 | 0.09227 | 0.01055 | 11.4 | F6 | 0.1043 | +18.7 | 0.0879 / 0.0846 | 2.7 | OK | False | |||
| 0.54 | 2 | E7/F7/G7 | 3 | 0.1211 | 0.007941 | 6.6 | F7 | 0.1302 | +11.1 | 0.1172 / 0.1158 | 0.8 | OK | False | |||
| 0.12 | 2 | E8/F8/G8 | 3 | 0.1322 | 0.009465 | 7.2 | E8 | 0.122 | -8.9 | 0.1407 / 0.1339 | 3.5 | OK | False | |||
| 0.026 | 2 | E9/F9/G9 | 3 | 0.1634 | 0.01667 | 10.2 | G9 | 0.1455 | -12.4 | 0.1661 / 0.1785 | 5.1 | OK | False | |||
| 0.006 | 2 | E10/F10/G10 | 3 | 0.1618 | 0.01004 | 6.2 | G10 | 0.1506 | -8.6 | 0.17 / 0.1648 | 2.2 | OK | False | |||
| 0.0006 | 2 | E11/F11/G11 | 3 | 0.169 | 0.03102 | 18.4 | F11 | 0.204 | +29.2 | 0.145 / 0.1579 | 6.0 | OK | True |
| Plate | CV_pct | Worst_Well | Worst_Value | Other_Reps | Worst_Dev_pct | CV_pct_drop1 | Verdict | Likely_Cause | Untreated | Reversal_Excluded | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Assay | Drug | Conc | |||||||||||
| Castor | 63GE8 | 1000 | 1 | 63.6 | D2 | 0.338 | 0.0886 / 0.1721 | +96.4 | 45.3 | scattered | False | True | |
| 222.2 | 1 | 64.9 | D3 | 0.1383 | 0.039 / 0.06247 | +121.4 | 32.7 | scattered | False | True | |||
| 49.4 | 1 | 26.2 | B4 | 0.03774 | 0.06464 / 0.05811 | -35.0 | 7.5 | one suspect well | Row B biased (-23% plate-wide) | False | False |
Mean % deviation of each plate row from its own triplicate median. Near 0 = that replicate position tracks the others. Beyond ±15% the position is systematically off, and individual wells flagged there are symptoms of that, not independent bad wells. The other plate axis is not shown: it carries concentration, so a “bias” along it would just be the dose response.
| B | C | D | E | F | G | ||
|---|---|---|---|---|---|---|---|
| Assay | Plate | ||||||
| Castor | 1 | -22.6 | +2.0 | +25.1 | -2.5 | -5.4 | +1.0 |
| 2 | -4.4 | +3.8 | -1.0 | +0.5 | -6.2 | -2.8 | |
| MTT | 1 | +4.9 | +3.6 | -8.3 | -1.4 | +11.0 | -3.1 |
| 2 | +1.1 | -5.9 | +1.8 | -1.4 | +12.2 | -5.2 |
Replicate groups: 80 (3–3 wells each)
Above 25% CV: 3 [1 with a single suspect well, 2 scattered]
Systematically biased replicate positions (|mean deviation| >= 15%):
Castor plate 1, row B: -22.6%
Castor plate 1, row D: +25.1%
-> re-check seeding/edge effects for these positions before blaming individual wells.
# ===========================================================================
# 4. Style mapping — COLOUR = drug, LINESTYLE/MARKER/HATCH = assay
# A drug keeps one colour in every plot and in every legend; the assay it was
# measured with is what changes the line (Castor dashed, MTT solid).
# ===========================================================================
all_drugs = sorted(df.Drug.dropna().unique())
all_assays = sorted(df.Assay.dropna().unique())
all_days = sorted(df.Day.unique())
# Day 0, if scanned, is a seeding-check timepoint and is not fitted.
fit_days = [d for d in all_days if d > 0] or all_days
# One series = one (Drug, Assay) pair actually present in the data.
drug_assay_pairs = sorted(
df[['Drug', 'Assay']].dropna().drop_duplicates()
.itertuples(index=False, name=None)
)
def series_label(drug, assay):
"""Legend/table label for one curve; the assay suffix is dropped if there is only one."""
return f'{drug} · {assay}' if len(all_assays) > 1 else str(drug)
def _split_ab_lp(name):
"""Split 'AB-LP' on the first dash. LP may contain further dashes."""
idx = name.find('-')
return (name[:idx], name[idx+1:]) if idx >= 0 else (name, None)
# Unique AB prefixes and LP suffixes (still used by the linker-payload trellises)
_all_abs_setup = sorted({_split_ab_lp(d)[0] for d in all_drugs})
_all_lps_setup = sorted({lp for d in all_drugs
for lp in [_split_ab_lp(d)[1]] if lp is not None})
# --- Colour: one per drug, stable across assays ---
_palette = (sns.color_palette('tab10', len(all_drugs)) if len(all_drugs) <= 10
else sns.color_palette('husl', len(all_drugs)))
drug_colors = {d: _palette[i] for i, d in enumerate(all_drugs)}
# --- Line style / marker / hatch: one per assay ---
# Fallback cycles for assays not named in ASSAY_STYLES. The dash patterns are
# deliberately coarse — matplotlib's default '--'/':' are hard to tell apart on
# a crowded log axis and vanish in a short legend sample.
_ls_cycle = ['-', (0, (6, 2)), (0, (1, 1.8)), (0, (8, 2, 1.5, 2)), (0, (3, 1.5)),
(0, (6, 2, 1.5, 2, 1.5, 2)), (0, (10, 3))]
_lw_cycle = [1.7, 2.2, 2.4, 2.2, 2.0, 2.2, 2.2]
_mk_cycle = ['o', 's', '^', 'D', 'v', 'P', 'X']
_ht_cycle = ['', '//', '\\\\', 'xx', '..', '++']
assay_linestyles, assay_linewidths, assay_markers, assay_hatches = {}, {}, {}, {}
for i, a in enumerate(all_assays):
_st = ASSAY_STYLES.get(a, {})
assay_linestyles[a] = _st.get('linestyle', _ls_cycle[i % len(_ls_cycle)])
assay_linewidths[a] = _st.get('linewidth', _lw_cycle[i % len(_lw_cycle)])
assay_markers[a] = _st.get('marker', _mk_cycle[i % len(_mk_cycle)])
assay_hatches[a] = _st.get('hatch', _ht_cycle[i % len(_ht_cycle)])
# --- Raw-metric y-axis split (see RAW_RIGHT_AXIS_ASSAY in the user input) ---
# Only meaningful with more than one assay in the data; a single-assay run keeps
# one left axis and these helpers become no-ops.
_raw_right_assay = (RAW_RIGHT_AXIS_ASSAY
if len(all_assays) > 1 and RAW_RIGHT_AXIS_ASSAY in all_assays
else None)
_raw_left_assays = [a for a in all_assays if a != _raw_right_assay]
def _raw_ylabel(assays):
"""Axis label for a raw-metric axis carrying `assays`."""
return ' / '.join(ASSAY_RAW_YLABEL.get(a, f'{a} raw metric') for a in assays)
def raw_metric_axes(ax, assays=None):
"""Axes to draw each assay's raw metric on, splitting off a twin if asked.
Returns `(axis_for, sides)`: a drug-agnostic assay -> axis map, and a list of
`(axis, spine_side, assays)` for the caller to scale and label. Each axis
ends up holding one assay's artists only, so autoscaling (or `set_ylim` from
that assay's rows) gives each readout the full plot height and the curves
overlap instead of stacking. `sides` has one entry when unsplit, two when
split, so `len(sides) > 1` is the "is this plot split" test.
"""
assays = list(all_assays) if assays is None else list(assays)
right = _raw_right_assay if (len(assays) > 1 and _raw_right_assay in assays) else None
if right is None:
return {a: ax for a in assays}, [(ax, 'left', assays)]
ax2 = ax.twinx()
ax2.grid(False) # one set of gridlines, owned by the left axis
return ({a: (ax2 if a == right else ax) for a in assays},
[(ax, 'left', [a for a in assays if a != right]),
(ax2, 'right', [right])])
def style_raw_axis(ax, side, assays, split):
"""Y-label for one side of a raw-metric plot, plus a spine cue when split.
With one assay per side the spine is drawn in that assay's line style, so the
axis itself repeats the linestyle=assay encoding the curves already use.
"""
ax.set_ylabel(_raw_ylabel(assays))
if split and len(assays) == 1:
sp = ax.spines[side]
sp.set_visible(True)
sp.set_linestyle(assay_linestyles[assays[0]])
sp.set_linewidth(1.4)
sp.set_color('0.35')
def assay_axis_labels(sides):
"""assay -> 'Name (left/right axis)' for a split plot; empty dict if unsplit."""
if len(sides) < 2:
return {}
return {a: f'{a} ({side} axis)' for _, side, assays in sides for a in assays}
LEGEND_SAMPLE_LW = 1.5 # see the note in style_legend_handles below
def style_legend_handles(drugs=None, assays=None, drug_title='Drug', assay_title='Assay',
assay_labels=None):
"""Factored legend: one colour swatch per drug + one line sample per assay.
Factored rather than one entry per (drug, assay) curve, so the legend states
the encoding itself instead of repeating every combination.
`assay_labels` overrides the text of individual assay entries (e.g. to note
which y-axis an assay is drawn against); assays left out keep their name.
"""
drugs = list(all_drugs) if drugs is None else list(drugs)
assays = list(all_assays) if assays is None else list(assays)
assay_labels = assay_labels or {}
handles = [Line2D([], [], linestyle='none', marker='', label=drug_title)]
handles += [Line2D([], [], color=drug_colors[d], lw=2.5, label=str(d)) for d in drugs]
if len(assays) > 1:
handles += [Line2D([], [], linestyle='none', marker='', label=assay_title)]
# NB: matplotlib scales dash patterns by line width, so the sample is
# drawn thin (not at the assay's plot width) or one dash fills the key.
handles += [Line2D([], [], color='0.35', lw=LEGEND_SAMPLE_LW,
linestyle=assay_linestyles[a], marker=assay_markers[a],
markersize=5, label=str(assay_labels.get(a, a)))
for a in assays]
return handles
def marker_key_handle(label, assays=None):
"""Legend key reading ○/◻ — one hollow marker per assay, slash-separated.
IC50 points are drawn with the assay's marker in the drug's colour, so the
key shows every shape in use rather than picking one drug's colour, and one
assay's marker, to stand for all of them.
"""
assays = list(all_assays) if assays is None else list(assays)
parts = []
for a in assays:
if parts: # separator between shapes
parts.append(Line2D([], [], marker='$/$', markersize=7,
color='black', linestyle='none',
label='_nolegend_'))
parts.append(Line2D([], [], marker=assay_markers[a], markersize=7,
markerfacecolor='none', markeredgecolor='black',
markeredgewidth=1.2, linestyle='none',
label='_nolegend_'))
parts[0].set_label(label) # label rides on the first part
return tuple(parts)
def draw_legend(ax, handles, **kw):
"""ax.legend for handle lists that may contain tuples (multi-marker keys)."""
labels = []
for h in handles:
if isinstance(h, tuple):
labels.append(next((x.get_label() for x in h
if not x.get_label().startswith('_')), ''))
else:
labels.append(h.get_label())
kw['handler_map'] = {tuple: HandlerTuple(ndivide=None, pad=0.1),
**kw.get('handler_map', {})}
kw.setdefault('handlelength', 3.4)
return ax.legend(handles, labels, **kw)
print(f'\n{len(all_drugs)} drugs: {all_drugs}')
print(f' AB prefixes ({len(_all_abs_setup)}): {_all_abs_setup}')
print(f' Linker-payloads ({len(_all_lps_setup)}): {_all_lps_setup}')
print(f' Assays ({len(all_assays)}): {all_assays}')
for a in all_assays:
_side = 'right' if a == _raw_right_assay else 'left'
print(f' {a:<8} linestyle={assay_linestyles[a]!r} marker={assay_markers[a]!r}'
+ (f' raw-metric axis={_side}' if _raw_right_assay else ''))
print(f' Series (drug × assay): {len(drug_assay_pairs)}')
for d, a in drug_assay_pairs:
print(f' {series_label(d, a)}')
4 drugs: ['63GE8', '64GE6', '64VPE6', 'IsoGE8']
AB prefixes (4): ['63GE8', '64GE6', '64VPE6', 'IsoGE8']
Linker-payloads (0): []
Assays (2): ['Castor', 'MTT']
Castor linestyle=(0, (6, 2)) marker='o' raw-metric axis=left
MTT linestyle='-' marker='s' raw-metric axis=right
Series (drug × assay): 8
63GE8 · Castor
63GE8 · MTT
64GE6 · Castor
64GE6 · MTT
64VPE6 · Castor
64VPE6 · MTT
IsoGE8 · Castor
IsoGE8 · MTT
# ===========================================================================
# 5. Growth kinetics plots (per drug, both assays overlaid)
# Colour = concentration here (the drug is fixed per figure);
# linestyle/marker still = assay.
# The metric is raw and its units differ by assay, so one assay is drawn
# against a twin right-hand axis (see raw_metric_axes).
# ===========================================================================
print('\n=== 4. Growth kinetics plots ===')
for drug in all_drugs:
sub_drug = df[df.Drug == drug]
if sub_drug.empty:
continue
assays_here = [a for a in all_assays if not sub_drug[sub_drug.Assay == a].empty]
concs = sorted(sub_drug.Conc.unique())
conc_colors = dict(zip(concs, sns.color_palette('viridis', len(concs))))
fig, ax = plt.subplots(figsize=(8, 5))
axis_for, sides = raw_metric_axes(ax, assays_here)
for assay in assays_here:
_ax = axis_for[assay]
for conc in concs:
csub = sub_drug[(sub_drug.Assay == assay) & (sub_drug.Conc == conc)]
if csub.empty:
continue
means = csub.groupby('Day')['Confluence'].mean()
sds = csub.groupby('Day')['Confluence'].std()
_ax.errorbar(means.index, means.values, yerr=sds.values,
marker=assay_markers[assay], markersize=5, capsize=2,
color=conc_colors[conc],
linestyle=assay_linestyles[assay],
linewidth=assay_linewidths[assay], elinewidth=0.9)
ax.set_xlabel('Day')
# Each side holds one assay's curves only, so matplotlib's own autoscaling
# already gives each readout the full plot height — no explicit limits.
_split = len(sides) > 1
for _ax, _side, _assays in sides:
style_raw_axis(_ax, _side, _assays, _split)
ax.set_xticks(all_days)
_axis_labels = assay_axis_labels(sides)
handles = [Line2D([], [], color=conc_colors[c], lw=2.5,
label='untreated' if c == PSEUDO_CONC else f'{c} nM')
for c in concs]
if len(assays_here) > 1:
handles += [Line2D([], [], linestyle='none', marker='', label='Assay')]
handles += [Line2D([], [], color='0.35', lw=LEGEND_SAMPLE_LW,
linestyle=assay_linestyles[a], marker=assay_markers[a],
markersize=5, label=str(_axis_labels.get(a, a)))
for a in assays_here]
# Legend on the topmost axis, so the twin's curves cannot be drawn over it.
draw_legend(sides[-1][0], handles, title='Conc', fontsize=8, title_fontsize=9)
fig.suptitle(f'{drug} — Growth Kinetics ({" + ".join(assays_here)})', fontsize=14)
plt.tight_layout()
save_fig()
=== 4. Growth kinetics plots ===
# ===========================================================================
# 6. LL.4 fitting functions
# ===========================================================================
print('\n=== 5. LL.4 model ===')
def ll4(x, slope, lower, upper, ec50):
return lower + (upper - lower) / (1.0 + (x / ec50) ** slope)
def compute_r2(y_obs, y_pred):
ss_res = np.sum((y_obs - y_pred) ** 2)
ss_tot = np.sum((y_obs - np.mean(y_obs)) ** 2)
if ss_tot == 0:
return 0.0
return 1.0 - ss_res / ss_tot
def fit_ll4(conc, response, p0=None):
"""Fit LL.4 using the points supplied by the shared analysis filter."""
conc = np.asarray(conc, dtype=float)
response = np.asarray(response, dtype=float)
mask = (conc > 0) & np.isfinite(response)
conc, response = conc[mask], response[mask]
if len(conc) < 4 or len(np.unique(conc)) < 3:
return None, None
cmin, cmax = conc.min(), conc.max()
if p0 is None:
p0 = [1.0, np.min(response), np.max(response), np.median(conc)]
bounds = (
[0.01, -np.inf, -np.inf, cmin * 0.01],
[np.inf, np.inf, np.inf, cmax * 100 ]
)
try:
popt, _ = curve_fit(ll4, conc, response, p0=p0,
maxfev=10000, bounds=bounds)
except Exception:
return None, None
y_pred = ll4(conc, *popt)
r2 = compute_r2(response, y_pred)
return popt, r2
def fit_ll4_pct_viable(conc, response, upper=100.0):
"""Fit %Viable LL.4 with the untreated-normalized upper plateau fixed at 100%."""
conc = np.asarray(conc, dtype=float)
response = np.asarray(response, dtype=float)
mask = (conc > 0) & np.isfinite(response)
conc, response = conc[mask], response[mask]
if len(conc) < 4 or len(np.unique(conc)) < 3:
return None, None
cmin, cmax = conc.min(), conc.max()
def _ll4_fixed_upper(x, slope, lower, ec50):
return ll4(x, slope, lower, upper, ec50)
try:
popt_fixed, _ = curve_fit(
_ll4_fixed_upper, conc, response,
p0=[1.0, np.min(response), np.median(conc)],
bounds=([0.01, -np.inf, cmin * 0.01],
[np.inf, np.inf, cmax * 100]),
maxfev=10000,
)
except Exception:
return None, None
slope, lower, ec50 = popt_fixed
popt = np.array([slope, lower, upper, ec50])
r2 = compute_r2(response, ll4(conc, *popt))
return popt, r2
def classify_ic50(row):
if row['R2'] < R2_THRESHOLD:
return 'poor fit'
if row['IC50_nM'] < CONC_MIN * 0.5:
return f'<{CONC_MIN} nM'
if row['IC50_nM'] > CONC_MAX * 2:
return f'>{CONC_MAX} nM'
return f"{row['IC50_nM']:.3g}"
def interpolate_ic50(concs, pct_viable, target=50.0):
"""
Find concentration where mean %Viable crosses `target` (default 50%).
Linear interpolation on log10(conc).
Exclusions are applied before this function is called.
Returns NaN if the curve never crosses the target.
"""
means = pct_viable.groupby(concs).mean()
c = np.array(means.index, dtype=float)
v = np.array(means.values, dtype=float)
order = np.argsort(c)
c, v = c[order], v[order]
for i in range(len(v) - 1):
if (v[i] >= target and v[i+1] < target) or \
(v[i] <= target and v[i+1] > target):
log_c1, log_c2 = np.log10(c[i]), np.log10(c[i+1])
frac = (target - v[i]) / (v[i+1] - v[i])
return 10 ** (log_c1 + frac * (log_c2 - log_c1))
if v[-1] < target and v[0] >= target:
return c[-1]
return np.nan
print(f'LL.4 defined. R² threshold = {R2_THRESHOLD}')
print('Interpolation IC50 function defined.')
print('Drug/day-specific >25% reversal exclusions active for all IC50 calculations.')
=== 5. LL.4 model ===
LL.4 defined. R² threshold = 0.5
Interpolation IC50 function defined.
Drug/day-specific >25% reversal exclusions active for all IC50 calculations.
# ===========================================================================
# 7. Dose-response fitting & plotting — per (Drug, Assay)
# Every fit, normalization and IC50 below is keyed on (Drug, Assay, Day):
# Castor and MTT measurements of the same drug are never pooled.
# ===========================================================================
metric_name = 'Confluence'
minfo = {
'col': 'Confluence',
'ylabel_pv': '% Viable (untreated-normalized, within assay)',
}
all_ic50_tables = {}
# x-axis window derived from user-defined scale bounds
_xlo = PSEUDO_CONC if SCALE_MIN_CONC == 0 else SCALE_MIN_CONC
_xhi = CONC_MAX if SCALE_MAX_CONC is None else SCALE_MAX_CONC
def _analysis_rows(sub_df):
"""Rows allowed to affect scaling, fitted curves, and IC50 calculations."""
return sub_df[~sub_df['Exclude_25pct_Reversal']].copy()
def _series_rows(drug, assay, day, value_col):
"""Wells for one (drug, assay, day) series with `value_col` present."""
sub = df[(df.Drug == drug) & (df.Assay == assay) & (df.Day == day)].copy()
return sub.dropna(subset=[value_col])
def _fit_window(sub_df):
"""Analysis-eligible rows inside the visible x-window."""
return _analysis_rows(sub_df[(sub_df.Conc >= _xlo) & (sub_df.Conc <= _xhi)])
def _axis_limits(sub_df, col, xlo, xhi, pad=0.10):
"""Y-limits from data within [xlo, xhi], honouring SCALE_EXCL_ABS."""
vis = _analysis_rows(sub_df[(sub_df['Conc'] >= xlo) & (sub_df['Conc'] <= xhi)])
if SCALE_EXCL_ABS and 'Drug' in vis.columns:
vis = vis[~vis['Drug'].str.split('-').str[0].isin(SCALE_EXCL_ABS)]
if vis.empty:
return None
m = vis.groupby('Conc')[col].mean()
s = vis.groupby('Conc')[col].std().fillna(0)
lo, hi = (m - s).min(), (m + s).max()
mg = (hi - lo) * pad if hi > lo else max(abs(hi), abs(lo)) * pad + 0.1
return lo - mg, hi + mg
def _plot_series(ax, drug, assay, sub, value_col, fit_params, day):
"""Points + LL.4 curve for one series. Colour = drug, linestyle/marker = assay.
With 8 series on one axis the SD whiskers otherwise dominate, so they are
drawn faint and behind the fitted curves; the markers stay full strength and
carry a thin white edge so overlapping points still read as separate.
"""
means = sub.groupby('Conc')[value_col].mean()
sds = sub.groupby('Conc')[value_col].std()
ax.errorbar(means.index, means.values, yerr=sds.values,
marker=assay_markers[assay], markersize=5,
markeredgecolor='white', markeredgewidth=0.6,
color=drug_colors[drug], linestyle='none',
ecolor=matplotlib.colors.to_rgba(drug_colors[drug], 0.40),
elinewidth=0.9, capsize=2, capthick=0.9, zorder=2)
key = (drug, assay, day)
if key not in fit_params:
return
curve_sub = _fit_window(sub)
if curve_sub.empty:
return
x_pred = np.geomspace(curve_sub.Conc.min(), curve_sub.Conc.max(), 200)
ax.plot(x_pred, ll4(x_pred, *fit_params[key]),
color=drug_colors[drug], linewidth=assay_linewidths[assay],
linestyle=assay_linestyles[assay],
solid_capstyle='round', dash_capstyle='round', zorder=3)
col = minfo['col']
print(f'\n{"="*60}')
print(f'=== METRIC: {metric_name} — assays: {", ".join(all_assays)} ===')
print(f'{"="*60}')
# --- 7a. Dose-response plots (raw metric) ---
print(f'\n--- Dose-response plots (raw {metric_name}) ---')
fit_params_dr = {}
ic50_dr_records = []
for drug, assay in drug_assay_pairs:
for day in fit_days:
sub = _fit_window(_series_rows(drug, assay, day, col).dropna(subset=['Conc']))
if sub.empty:
continue
popt, r2 = fit_ll4(sub.Conc.values, sub[col].values)
rec = {'Drug': drug, 'Assay': assay, 'Day': day}
if popt is not None:
fit_params_dr[(drug, assay, day)] = popt
rec.update({'IC50_nM': popt[3], 'Slope': popt[0],
'Lower': popt[1], 'Upper': popt[2], 'R2': r2})
else:
rec.update({'IC50_nM': np.nan, 'Slope': np.nan,
'Lower': np.nan, 'Upper': np.nan, 'R2': np.nan})
ic50_dr_records.append(rec)
for day in fit_days:
fig, ax = plt.subplots(figsize=(12, 7))
axis_for, sides = raw_metric_axes(ax)
_split = len(sides) > 1
for drug, assay in drug_assay_pairs:
sub = _series_rows(drug, assay, day, col)
if sub.empty:
continue
_plot_series(axis_for[assay], drug, assay, sub, col, fit_params_dr, day)
# Each side is scaled from its own assays' rows, so both readouts fill the
# plot height and overlap instead of stacking into separate bands.
for _ax, _side, _assays in sides:
_vis = df[(df.Day == day) & df.Assay.isin(_assays)].dropna(subset=[col])
_lim = _axis_limits(_vis, col, _xlo, _xhi)
if _lim:
_ax.set_ylim(*_lim)
style_raw_axis(_ax, _side, _assays, _split)
ax.set_xscale('log')
ax.set_xlabel('Concentration (nM, log scale)')
ax.set_title(
f'Day {day} — Dose Response (raw metric, one axis per assay: '
f'{"/".join(_raw_left_assays)} left, {_raw_right_assay} right)' if _split
else f'Day {day} — Dose Response (raw metric, units differ by assay)')
# Legend on the topmost axis, so the twin's curves cannot be drawn over it.
draw_legend(sides[-1][0],
style_legend_handles(assay_labels=assay_axis_labels(sides)),
fontsize=8, loc='lower left', ncol=2)
plt.tight_layout()
save_fig()
df_ic50_dr = pd.DataFrame(ic50_dr_records)
df_ic50_dr['Reliable'] = (
(df_ic50_dr['R2'] >= R2_THRESHOLD) &
(df_ic50_dr['IC50_nM'] >= CONC_MIN * 0.5) &
(df_ic50_dr['IC50_nM'] <= CONC_MAX * 2)
)
df_ic50_dr['Status'] = df_ic50_dr.apply(classify_ic50, axis=1)
print(f'\n IC50 ({metric_name} direct): {df_ic50_dr.Reliable.sum()} reliable '
f'of {len(df_ic50_dr)}')
# --- 7b. %Viable calculation — untreated baseline per Assay/Plate/Drug/Day ---
# Normalizing within the plate keeps each assay on its own baseline, which is
# what makes the two readouts comparable on one %Viable axis.
pv_col = f'PctViable_{metric_name}'
untx_col = f'UntxMean_{metric_name}'
untx_mean = (
df[df.Conc_orig == 0]
.groupby(['Assay', 'Plate', 'Drug', 'Day'])[col]
.mean()
.reset_index()
.rename(columns={col: untx_col})
)
df = df.drop(columns=[untx_col, pv_col], errors='ignore') # idempotent re-run
df = df.merge(untx_mean, on=['Assay', 'Plate', 'Drug', 'Day'], how='left')
df[pv_col] = 100.0 * df[col] / df[untx_col]
# --- 7c. Absolute IC50: interpolated crossing at 50% of untreated ---
interp_records = []
for drug, assay in drug_assay_pairs:
for day in fit_days:
sub = _fit_window(_series_rows(drug, assay, day, pv_col).dropna(subset=['Conc']))
if sub.empty:
continue
ic50_interp = interpolate_ic50(sub.Conc, sub[pv_col], target=50.0)
interp_records.append({
'Drug': drug, 'Assay': assay, 'Day': day,
'Absolute_IC50_nM': ic50_interp, 'IC50_interp': ic50_interp,
})
df_interp = pd.DataFrame(interp_records)
all_ic50_tables[f'{metric_name}_interpolated'] = df_interp
print(f'\n Absolute IC50 at 50% untreated ({metric_name}): '
f'{df_interp.IC50_interp.notna().sum()} of {len(df_interp)} crossed 50%')
# --- 7d. Fitted IC50: LL.4 midpoint between fitted upper/lower plateaus ---
fit_params_pv = {}
ic50_pv_records = []
for drug, assay in drug_assay_pairs:
for day in fit_days:
sub = _fit_window(_series_rows(drug, assay, day, pv_col).dropna(subset=['Conc']))
if sub.empty:
continue
popt, r2 = fit_ll4_pct_viable(sub.Conc.values, sub[pv_col].values)
rec = {'Drug': drug, 'Assay': assay, 'Day': day}
if popt is not None:
fit_params_pv[(drug, assay, day)] = popt
rec.update({'IC50_nM': popt[3], 'Fitted_IC50_nM': popt[3],
'Midpoint_Response': (popt[1] + popt[2]) / 2.0,
'Slope': popt[0], 'Lower': popt[1], 'Upper': popt[2], 'R2': r2})
else:
rec.update({'IC50_nM': np.nan, 'Fitted_IC50_nM': np.nan,
'Midpoint_Response': np.nan, 'Slope': np.nan,
'Lower': np.nan, 'Upper': np.nan, 'R2': np.nan})
ic50_pv_records.append(rec)
# --- 7e. %Viable dose-response plots with absolute IC50 markers ---
for day in fit_days:
_all_vis_pv = df[df.Day == day].dropna(subset=[pv_col])
_ylim_pv = _axis_limits(_all_vis_pv, pv_col, _xlo, _xhi)
if _ylim_pv:
_ylim_pv = (0, _ylim_pv[1])
fig, ax = plt.subplots(figsize=(12, 7))
_marked = False
for drug, assay in drug_assay_pairs:
sub = _series_rows(drug, assay, day, pv_col)
if sub.empty:
continue
_plot_series(ax, drug, assay, sub, pv_col, fit_params_pv, day)
row = df_interp[(df_interp.Drug == drug) & (df_interp.Assay == assay) &
(df_interp.Day == day)]
if not row.empty and pd.notna(row.IC50_interp.values[0]):
ax.plot(row.IC50_interp.values[0], 50, assay_markers[assay],
color=drug_colors[drug], markersize=11,
markeredgecolor='black', markeredgewidth=1.2, zorder=6)
_marked = True
ax.set_xscale('log')
if _ylim_pv:
ax.set_ylim(*_ylim_pv)
ax.set_xlabel('Concentration (nM, log scale)')
ax.set_ylabel(minfo['ylabel_pv'])
ax.set_title(f'Day {day} — %Viable — Absolute IC50 at 50% untreated')
ax.axhline(50, color='red', ls='--', alpha=0.7)
ax.axhline(100, color='grey', ls='--', alpha=0.3)
_extra = [Line2D([], [], color='red', ls='--', alpha=0.7, label='50% viable')]
if _marked:
_extra.append(marker_key_handle('Absolute IC50 (50% untx)'))
draw_legend(ax, style_legend_handles() + _extra,
fontsize=8, loc='lower left', ncol=2)
plt.tight_layout()
save_fig()
# --- 7f. %Viable dose-response plots with fitted-midpoint IC50 markers ---
for day in fit_days:
_all_vis_pv = df[df.Day == day].dropna(subset=[pv_col])
_ylim_pv = _axis_limits(_all_vis_pv, pv_col, _xlo, _xhi, pad=0.05)
fig, ax = plt.subplots(figsize=(12, 7))
_marked = False
for drug, assay in drug_assay_pairs:
sub = _series_rows(drug, assay, day, pv_col)
if sub.empty:
continue
_plot_series(ax, drug, assay, sub, pv_col, fit_params_pv, day)
key = (drug, assay, day)
if key in fit_params_pv:
popt = fit_params_pv[key]
fitted_ic50 = popt[3]
fitted_midpoint = (popt[1] + popt[2]) / 2.0
if _xlo <= fitted_ic50 <= _xhi:
ax.plot(fitted_ic50, fitted_midpoint, assay_markers[assay],
color=drug_colors[drug], markersize=11,
markeredgecolor='black', markeredgewidth=1.2, zorder=6)
_marked = True
ax.set_xscale('log')
if _ylim_pv:
ax.set_ylim(*_ylim_pv)
ax.set_xlabel('Concentration (nM, log scale)')
ax.set_ylabel(minfo['ylabel_pv'])
ax.set_title(f'Day {day} — %Viable — Fitted IC50 at LL.4 plateau midpoint')
_extra = []
if _marked:
_extra.append(marker_key_handle('Fitted IC50 (LL.4 midpoint)'))
draw_legend(ax, style_legend_handles() + _extra,
fontsize=8, loc='lower left', ncol=2)
plt.tight_layout()
save_fig()
df_ic50_pv = pd.DataFrame(ic50_pv_records)
df_ic50_pv['Reliable'] = (
(df_ic50_pv['R2'] >= R2_THRESHOLD) &
(df_ic50_pv['IC50_nM'] >= CONC_MIN * 0.5) &
(df_ic50_pv['IC50_nM'] <= CONC_MAX * 2)
)
df_ic50_pv['Status'] = df_ic50_pv.apply(classify_ic50, axis=1)
print(f' Fitted-midpoint IC50 (%Viable {metric_name}): {df_ic50_pv.Reliable.sum()} reliable '
f'of {len(df_ic50_pv)}')
all_ic50_tables[f'{metric_name}_direct'] = df_ic50_dr
all_ic50_tables[f'{metric_name}_pctViable'] = df_ic50_pv
============================================================
=== METRIC: Confluence — assays: Castor, MTT ===
============================================================
--- Dose-response plots (raw Confluence) ---
IC50 (Confluence direct): 7 reliable of 8
Absolute IC50 at 50% untreated (Confluence): 6 of 8 crossed 50%
Fitted-midpoint IC50 (%Viable Confluence): 6 reliable of 8
# ===========================================================================
# 7g. Dose-response fitting & plotting — suspect replicates DROPPED
# Same LL.4 model, same normalization, same plots as section 7, re-run on a
# copy of the data with the wells section 3b flagged removed first.
#
# Section 7 is deliberately left intact: nothing here rebinds `df`,
# `fit_params_pv`, `df_ic50_pv`, `df_interp` or `all_ic50_tables`, which the
# LL.5 fit, the trellises, the summary tables and the export all read. This
# is a parallel view to compare against, not a replacement — so the two can
# be read side by side and the effect of dropping is visible rather than
# assumed.
# ===========================================================================
print('\n=== 7g. Dose-response with suspect replicates dropped ===')
# --- USER INPUT: what counts as droppable ----------------------------------
DROP_SUSPECT_WELLS = True # 'one suspect well' groups: drop that well,
# leaving the two replicates that agree.
SCATTERED_POLICY = 'drop_worst' # 'scattered' groups — no single well explains
# the spread, so choose how to treat them:
# 'drop_group' drop all of its wells; that
# concentration is lost from
# the curve entirely
# 'drop_worst' drop only the well furthest
# from the median, leaving n=2
# 'keep' leave the group untouched
REGISTER_QC_TABLES_IN_EXPORTS = False # True also feeds these IC50s into the
# section-8 summary tables and the
# section-10 CSV export. Off by default
# so this section changes nothing
# downstream.
# ---------------------------------------------------------------------------
assert 'df_replicate_qc' in globals(), 'Run section 3b (replicate variance) first.'
assert 'pv_col' in globals(), 'Run section 7 (dose-response fitting) first.'
assert SCATTERED_POLICY in ('drop_group', 'drop_worst', 'keep'), \
f'SCATTERED_POLICY must be drop_group/drop_worst/keep, got {SCATTERED_POLICY!r}'
# --- Which wells does the section-3b verdict remove? -----------------------
# A well is addressed by (Assay, Plate, Day, Well): plate numbers repeat across
# assays and well IDs repeat across plates, so `Well` alone identifies nothing.
_drop_keys, _drop_log = set(), []
for _r in df_replicate_qc.itertuples(index=False):
if _r.Verdict == 'one suspect well' and DROP_SUSPECT_WELLS:
_wells, _why = [_r.Suspect_Well], 'suspect well'
elif _r.Verdict == 'scattered' and SCATTERED_POLICY == 'drop_group':
_wells, _why = _r.Wells.split('/'), 'scattered — whole group'
elif _r.Verdict == 'scattered' and SCATTERED_POLICY == 'drop_worst':
_wells, _why = [_r.Worst_Well], 'scattered — worst well'
else:
continue
for _w in _wells:
_drop_keys.add((_r.Assay, _r.Plate, _r.Day, _w))
_drop_log.append({
'Assay': _r.Assay, 'Plate': _r.Plate, 'Drug': _r.Drug, 'Day': _r.Day,
'Conc': _r.Conc, 'Well': _w, 'CV_pct': _r.CV_pct,
'CV_pct_drop1': _r.CV_pct_drop1, 'Verdict': _r.Verdict,
'Reason': _why, 'Untreated': _r.Untreated,
'Likely_Cause': _r.Likely_Cause,
})
df_dropped_wells = pd.DataFrame(_drop_log)
# Section 3b judges every well, including wells at concentrations the 25%
# reversal filter or the x-window had already removed from the fit. Dropping one
# of those changes nothing, so mark which dropped wells were actually reaching
# the curve — otherwise a series shows wells dropped and a zero IC50 shift and
# looks like a bug.
_fit_wells = set(zip(*(_fit_window(df)[c] for c in
['Assay', 'Plate', 'Day', 'Well'])))
if not df_dropped_wells.empty:
df_dropped_wells['In_Fit_Window'] = [
k in _fit_wells for k in zip(df_dropped_wells.Assay, df_dropped_wells.Plate,
df_dropped_wells.Day, df_dropped_wells.Well)
]
# --- The cleaned copy -------------------------------------------------------
# `df` itself is never modified — every later section still sees all wells.
df_clean = df.copy()
df_clean['Dropped_Replicate_QC'] = [
k in _drop_keys
for k in zip(df_clean.Assay, df_clean.Plate, df_clean.Day, df_clean.Well)
]
_n_before = len(df_clean)
df_clean = df_clean[~df_clean.Dropped_Replicate_QC].copy()
_n_dropped = _n_before - len(df_clean)
_n_effective = (0 if df_dropped_wells.empty
else int(df_dropped_wells.In_Fit_Window.sum()))
print(f' Flagged groups in 3b : {(df_replicate_qc.Verdict != "OK").sum()} '
f'of {len(df_replicate_qc)}')
print(f' Wells dropped : {_n_dropped} of {_n_before} '
f'({100.0 * _n_dropped / _n_before:.1f}%)')
print(f' ...reaching the fit : {_n_effective} '
f'({_n_dropped - _n_effective} were already outside it — reversal filter '
f'or x-window — so they cannot move any curve)')
print(f' Policy : suspect wells={"drop" if DROP_SUSPECT_WELLS else "keep"}, '
f'scattered={SCATTERED_POLICY}')
if df_dropped_wells.empty:
display(Markdown('_No wells met the drop criteria — the curves below are '
'identical to section 7._'))
else:
display(Markdown('### Wells dropped before fitting'))
display(Markdown(
'`In_Fit_Window` is whether the well was still reaching the section-7 '
'fit; a `False` here means the reversal filter had already removed its '
'concentration, so dropping it cannot change any IC50.'))
display(
df_dropped_wells.sort_values(['Assay', 'Plate', 'Drug', 'Conc'])
.style.hide(axis='index')
.format({'CV_pct': '{:.1f}', 'CV_pct_drop1': '{:.1f}', 'Conc': '{:g}'},
na_rep='—')
.map(lambda v: 'color: #757575' if v == False else None,
subset=['In_Fit_Window'])
)
# --- %Viable, recomputed on the cleaned data --------------------------------
# This cannot be inherited from section 7. If a dropped well was an untreated
# well, the baseline that every point on that curve is divided by has changed,
# so %Viable must be rebuilt from the surviving wells. The columns keep their
# section-7 names because this is a different DataFrame — `df` is untouched.
_untx_clean = (
df_clean[df_clean.Conc_orig == 0]
.groupby(['Assay', 'Plate', 'Drug', 'Day'])[col]
.mean().reset_index().rename(columns={col: untx_col})
)
df_clean = df_clean.drop(columns=[untx_col, pv_col], errors='ignore')
df_clean = df_clean.merge(_untx_clean, on=['Assay', 'Plate', 'Drug', 'Day'], how='left')
df_clean[pv_col] = 100.0 * df_clean[col] / df_clean[untx_col]
def _series_rows_clean(drug, assay, day, value_col):
"""`_series_rows`, reading the QC-cleaned frame instead of `df`."""
sub = df_clean[(df_clean.Drug == drug) & (df_clean.Assay == assay) &
(df_clean.Day == day)].copy()
return sub.dropna(subset=[value_col])
# Every other section-7 helper (`_analysis_rows`, `_fit_window`, `_axis_limits`,
# `_plot_series`) already takes its rows as an argument, so they are reused as
# they are — the reversal filter, the x-window and the plot styling stay
# identical and only the underlying wells differ.
_QC_TAG = 'suspect replicates dropped'
# --- 7g-a. Dose-response fits & plots (raw metric) --------------------------
print(f'\n--- Dose-response plots (raw {metric_name}, {_QC_TAG}) ---')
fit_params_dr_qc = {}
ic50_dr_qc_records = []
for drug, assay in drug_assay_pairs:
for day in fit_days:
sub = _fit_window(_series_rows_clean(drug, assay, day, col).dropna(subset=['Conc']))
if sub.empty:
continue
popt, r2 = fit_ll4(sub.Conc.values, sub[col].values)
rec = {'Drug': drug, 'Assay': assay, 'Day': day}
if popt is not None:
fit_params_dr_qc[(drug, assay, day)] = popt
rec.update({'IC50_nM': popt[3], 'Slope': popt[0],
'Lower': popt[1], 'Upper': popt[2], 'R2': r2})
else:
rec.update({'IC50_nM': np.nan, 'Slope': np.nan,
'Lower': np.nan, 'Upper': np.nan, 'R2': np.nan})
ic50_dr_qc_records.append(rec)
for day in fit_days:
fig, ax = plt.subplots(figsize=(12, 7))
axis_for, sides = raw_metric_axes(ax)
_split = len(sides) > 1
for drug, assay in drug_assay_pairs:
sub = _series_rows_clean(drug, assay, day, col)
if sub.empty:
continue
_plot_series(axis_for[assay], drug, assay, sub, col, fit_params_dr_qc, day)
for _ax, _side, _assays in sides:
_vis = df_clean[(df_clean.Day == day) & df_clean.Assay.isin(_assays)].dropna(subset=[col])
_lim = _axis_limits(_vis, col, _xlo, _xhi)
if _lim:
_ax.set_ylim(*_lim)
style_raw_axis(_ax, _side, _assays, _split)
ax.set_xscale('log')
ax.set_xlabel('Concentration (nM, log scale)')
ax.set_title(f'Day {day} — Dose Response (raw metric, {_QC_TAG})')
draw_legend(sides[-1][0],
style_legend_handles(assay_labels=assay_axis_labels(sides)),
fontsize=8, loc='lower left', ncol=2)
plt.tight_layout()
save_fig()
df_ic50_dr_qc = pd.DataFrame(ic50_dr_qc_records)
df_ic50_dr_qc['Reliable'] = (
(df_ic50_dr_qc['R2'] >= R2_THRESHOLD) &
(df_ic50_dr_qc['IC50_nM'] >= CONC_MIN * 0.5) &
(df_ic50_dr_qc['IC50_nM'] <= CONC_MAX * 2)
)
df_ic50_dr_qc['Status'] = df_ic50_dr_qc.apply(classify_ic50, axis=1)
print(f'\n IC50 ({metric_name} direct, {_QC_TAG}): '
f'{df_ic50_dr_qc.Reliable.sum()} reliable of {len(df_ic50_dr_qc)}')
# --- 7g-b. Absolute IC50: interpolated crossing at 50% of untreated ---------
interp_qc_records = []
for drug, assay in drug_assay_pairs:
for day in fit_days:
sub = _fit_window(_series_rows_clean(drug, assay, day, pv_col).dropna(subset=['Conc']))
if sub.empty:
continue
ic50_interp = interpolate_ic50(sub.Conc, sub[pv_col], target=50.0)
interp_qc_records.append({
'Drug': drug, 'Assay': assay, 'Day': day,
'Absolute_IC50_nM': ic50_interp, 'IC50_interp': ic50_interp,
})
df_interp_qc = pd.DataFrame(interp_qc_records)
print(f' Absolute IC50 at 50% untreated ({_QC_TAG}): '
f'{df_interp_qc.IC50_interp.notna().sum()} of {len(df_interp_qc)} crossed 50%')
# --- 7g-c. Fitted IC50: LL.4 midpoint between fitted plateaus ---------------
fit_params_pv_qc = {}
ic50_pv_qc_records = []
for drug, assay in drug_assay_pairs:
for day in fit_days:
sub = _fit_window(_series_rows_clean(drug, assay, day, pv_col).dropna(subset=['Conc']))
if sub.empty:
continue
popt, r2 = fit_ll4_pct_viable(sub.Conc.values, sub[pv_col].values)
rec = {'Drug': drug, 'Assay': assay, 'Day': day}
if popt is not None:
fit_params_pv_qc[(drug, assay, day)] = popt
rec.update({'IC50_nM': popt[3], 'Fitted_IC50_nM': popt[3],
'Midpoint_Response': (popt[1] + popt[2]) / 2.0,
'Slope': popt[0], 'Lower': popt[1], 'Upper': popt[2], 'R2': r2})
else:
rec.update({'IC50_nM': np.nan, 'Fitted_IC50_nM': np.nan,
'Midpoint_Response': np.nan, 'Slope': np.nan,
'Lower': np.nan, 'Upper': np.nan, 'R2': np.nan})
ic50_pv_qc_records.append(rec)
# --- 7g-d. %Viable plots with absolute IC50 markers ------------------------
for day in fit_days:
_all_vis_pv = df_clean[df_clean.Day == day].dropna(subset=[pv_col])
_ylim_pv = _axis_limits(_all_vis_pv, pv_col, _xlo, _xhi)
if _ylim_pv:
_ylim_pv = (0, _ylim_pv[1])
fig, ax = plt.subplots(figsize=(12, 7))
_marked = False
for drug, assay in drug_assay_pairs:
sub = _series_rows_clean(drug, assay, day, pv_col)
if sub.empty:
continue
_plot_series(ax, drug, assay, sub, pv_col, fit_params_pv_qc, day)
row = df_interp_qc[(df_interp_qc.Drug == drug) & (df_interp_qc.Assay == assay) &
(df_interp_qc.Day == day)]
if not row.empty and pd.notna(row.IC50_interp.values[0]):
ax.plot(row.IC50_interp.values[0], 50, assay_markers[assay],
color=drug_colors[drug], markersize=11,
markeredgecolor='black', markeredgewidth=1.2, zorder=6)
_marked = True
ax.set_xscale('log')
if _ylim_pv:
ax.set_ylim(*_ylim_pv)
ax.set_xlabel('Concentration (nM, log scale)')
ax.set_ylabel(minfo['ylabel_pv'])
ax.set_title(f'Day {day} — %Viable — Absolute IC50 at 50% untreated ({_QC_TAG})')
ax.axhline(50, color='red', ls='--', alpha=0.7)
ax.axhline(100, color='grey', ls='--', alpha=0.3)
_extra = [Line2D([], [], color='red', ls='--', alpha=0.7, label='50% viable')]
if _marked:
_extra.append(marker_key_handle('Absolute IC50 (50% untx)'))
draw_legend(ax, style_legend_handles() + _extra,
fontsize=8, loc='lower left', ncol=2)
plt.tight_layout()
save_fig()
# --- 7g-e. %Viable plots with fitted-midpoint IC50 markers -----------------
for day in fit_days:
_all_vis_pv = df_clean[df_clean.Day == day].dropna(subset=[pv_col])
_ylim_pv = _axis_limits(_all_vis_pv, pv_col, _xlo, _xhi, pad=0.05)
fig, ax = plt.subplots(figsize=(12, 7))
_marked = False
for drug, assay in drug_assay_pairs:
sub = _series_rows_clean(drug, assay, day, pv_col)
if sub.empty:
continue
_plot_series(ax, drug, assay, sub, pv_col, fit_params_pv_qc, day)
key = (drug, assay, day)
if key in fit_params_pv_qc:
popt = fit_params_pv_qc[key]
fitted_ic50 = popt[3]
fitted_midpoint = (popt[1] + popt[2]) / 2.0
if _xlo <= fitted_ic50 <= _xhi:
ax.plot(fitted_ic50, fitted_midpoint, assay_markers[assay],
color=drug_colors[drug], markersize=11,
markeredgecolor='black', markeredgewidth=1.2, zorder=6)
_marked = True
ax.set_xscale('log')
if _ylim_pv:
ax.set_ylim(*_ylim_pv)
ax.set_xlabel('Concentration (nM, log scale)')
ax.set_ylabel(minfo['ylabel_pv'])
ax.set_title(f'Day {day} — %Viable — Fitted IC50 at LL.4 plateau midpoint ({_QC_TAG})')
_extra = []
if _marked:
_extra.append(marker_key_handle('Fitted IC50 (LL.4 midpoint)'))
draw_legend(ax, style_legend_handles() + _extra,
fontsize=8, loc='lower left', ncol=2)
plt.tight_layout()
save_fig()
df_ic50_pv_qc = pd.DataFrame(ic50_pv_qc_records)
df_ic50_pv_qc['Reliable'] = (
(df_ic50_pv_qc['R2'] >= R2_THRESHOLD) &
(df_ic50_pv_qc['IC50_nM'] >= CONC_MIN * 0.5) &
(df_ic50_pv_qc['IC50_nM'] <= CONC_MAX * 2)
)
df_ic50_pv_qc['Status'] = df_ic50_pv_qc.apply(classify_ic50, axis=1)
print(f' Fitted-midpoint IC50 (%Viable, {_QC_TAG}): '
f'{df_ic50_pv_qc.Reliable.sum()} reliable of {len(df_ic50_pv_qc)}')
all_ic50_tables_qc = {
f'{metric_name}_direct_QCdropped': df_ic50_dr_qc,
f'{metric_name}_pctViable_QCdropped': df_ic50_pv_qc,
f'{metric_name}_interpolated_QCdropped': df_interp_qc,
}
if REGISTER_QC_TABLES_IN_EXPORTS:
all_ic50_tables.update(all_ic50_tables_qc)
print(f' Registered {len(all_ic50_tables_qc)} QC tables into all_ic50_tables '
f'(they will appear in the section-8 summaries and section-10 export).')
# --- 7g-f. What did dropping actually change? ------------------------------
# The point of the section: the two IC50 sets side by side. A large shift means
# the fit was leaning on a well that section 3b called suspect.
_KEYS = ['Drug', 'Assay', 'Day']
def _delta_pct(new, old):
return np.where((old > 0) & np.isfinite(new), 100.0 * (new - old) / old, np.nan)
_cmp_fit = (
df_ic50_pv[_KEYS + ['IC50_nM', 'R2', 'Reliable']]
.rename(columns={'IC50_nM': 'IC50_all', 'R2': 'R2_all', 'Reliable': 'Rel_all'})
.merge(df_ic50_pv_qc[_KEYS + ['IC50_nM', 'R2', 'Reliable']]
.rename(columns={'IC50_nM': 'IC50_QC', 'R2': 'R2_QC', 'Reliable': 'Rel_QC'}),
on=_KEYS, how='outer')
)
_cmp_fit['IC50_delta_%'] = _delta_pct(_cmp_fit.IC50_QC, _cmp_fit.IC50_all)
_cmp_fit['R2_delta'] = _cmp_fit.R2_QC - _cmp_fit.R2_all
_cmp_int = (
df_interp[_KEYS + ['IC50_interp']].rename(columns={'IC50_interp': 'Abs_all'})
.merge(df_interp_qc[_KEYS + ['IC50_interp']].rename(columns={'IC50_interp': 'Abs_QC'}),
on=_KEYS, how='outer')
)
_cmp_int['Abs_delta_%'] = _delta_pct(_cmp_int.Abs_QC, _cmp_int.Abs_all)
_cmp = _cmp_fit.merge(_cmp_int, on=_KEYS, how='outer')
# How many wells each series lost, and how many of those were reaching the fit.
# `Dropped_in_fit == 0` must come with a zero delta — that is the sanity check on
# the whole section. `Wells_dropped` alone does not: a series can lose a whole
# group at a concentration the reversal filter had already excluded and be
# completely unaffected.
_lost_all = ({} if df_dropped_wells.empty
else df_dropped_wells.groupby(_KEYS).size().to_dict())
_lost_fit = ({} if df_dropped_wells.empty
else df_dropped_wells[df_dropped_wells.In_Fit_Window]
.groupby(_KEYS).size().to_dict())
_cmp['Wells_dropped'] = [_lost_all.get(k, 0) for k in
zip(_cmp.Drug, _cmp.Assay, _cmp.Day)]
_cmp['Dropped_in_fit'] = [_lost_fit.get(k, 0) for k in
zip(_cmp.Drug, _cmp.Assay, _cmp.Day)]
display(Markdown('### IC50 before vs after dropping suspect replicates'))
display(Markdown(
'`IC50_all` / `Abs_all` are section 7 (every well); `_QC` are the same '
'quantities refitted without the dropped wells. `Dropped_in_fit` is the '
'number of dropped wells that were still reaching the fit — a series with '
'`0` there must show a delta of `+0.0`, and a large delta where only one '
'well went means the LL.4 fit was being steered by that single well. '
'`Abs_delta_%` moves furthest when a whole concentration is dropped, since '
'the 50% crossing then has to be interpolated across a wider gap.'))
display(
_cmp.sort_values('IC50_delta_%', key=lambda s: s.abs(), ascending=False)
.style.hide(axis='index')
.format({'IC50_all': '{:.4g}', 'IC50_QC': '{:.4g}', 'IC50_delta_%': '{:+.1f}',
'R2_all': '{:.3f}', 'R2_QC': '{:.3f}', 'R2_delta': '{:+.3f}',
'Abs_all': '{:.4g}', 'Abs_QC': '{:.4g}', 'Abs_delta_%': '{:+.1f}'},
na_rep='—')
.map(lambda v: 'font-weight: bold' if isinstance(v, (int, float, np.floating))
and np.isfinite(v) and abs(v) >= 20 else None,
subset=['IC50_delta_%', 'Abs_delta_%'])
.map(lambda v: 'color: #b00020' if v == False else None, subset=['Rel_all', 'Rel_QC'])
)
print(f'\n Series whose fitted IC50 moved >20%: '
f'{int((_cmp["IC50_delta_%"].abs() > 20).sum())} of {len(_cmp)}')
print(f' Series that became reliable: '
f'{int((~_cmp.Rel_all.fillna(False) & _cmp.Rel_QC.fillna(False)).sum())}; '
f'lost reliability: '
f'{int((_cmp.Rel_all.fillna(False) & ~_cmp.Rel_QC.fillna(False)).sum())}')
print(' `df` and every section-7 table are unchanged — later sections still use all wells.')
=== 7g. Dose-response with suspect replicates dropped ===
Flagged groups in 3b : 3 of 80
Wells dropped : 3 of 240 (1.2%)
...reaching the fit : 1 (2 were already outside it — reversal filter or x-window — so they cannot move any curve)
Policy : suspect wells=drop, scattered=drop_worst
In_Fit_Window is whether the well was still reaching the section-7 fit; a False here means the reversal filter had already removed its concentration, so dropping it cannot change any IC50.
| Assay | Plate | Drug | Day | Conc | Well | CV_pct | CV_pct_drop1 | Verdict | Reason | Untreated | Likely_Cause | In_Fit_Window |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Castor | 1 | 63GE8 | 7 | 49.4 | B4 | 26.2 | 7.5 | one suspect well | suspect well | False | Row B biased (-23% plate-wide) | True |
| Castor | 1 | 63GE8 | 7 | 222.2 | D3 | 64.9 | 32.7 | scattered | scattered — worst well | False | False | |
| Castor | 1 | 63GE8 | 7 | 1000 | D2 | 63.6 | 45.3 | scattered | scattered — worst well | False | False |
--- Dose-response plots (raw Confluence, suspect replicates dropped) ---
IC50 (Confluence direct, suspect replicates dropped): 7 reliable of 8
Absolute IC50 at 50% untreated (suspect replicates dropped): 6 of 8 crossed 50%
Fitted-midpoint IC50 (%Viable, suspect replicates dropped): 6 reliable of 8
IC50_all / Abs_all are section 7 (every well); _QC are the same quantities refitted without the dropped wells. Dropped_in_fit is the number of dropped wells that were still reaching the fit — a series with 0 there must show a delta of +0.0, and a large delta where only one well went means the LL.4 fit was being steered by that single well. Abs_delta_% moves furthest when a whole concentration is dropped, since the 50% crossing then has to be interpolated across a wider gap.
| Drug | Assay | Day | IC50_all | R2_all | Rel_all | IC50_QC | R2_QC | Rel_QC | IC50_delta_% | R2_delta | Abs_all | Abs_QC | Abs_delta_% | Wells_dropped | Dropped_in_fit |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 63GE8 | Castor | 7 | 0.08313 | 0.578 | True | 0.0688 | 0.584 | True | -17.2 | +0.006 | — | — | — | 3 | 1 |
| 63GE8 | MTT | 7 | 0.1366 | 0.905 | True | 0.1366 | 0.905 | True | +0.0 | +0.000 | 22.69 | 22.69 | +0.0 | 0 | 0 |
| 64GE6 | Castor | 7 | 0.09133 | 0.911 | True | 0.09133 | 0.911 | True | +0.0 | +0.000 | 0.8566 | 0.8566 | +0.0 | 0 | 0 |
| 64GE6 | MTT | 7 | 0.2644 | 0.934 | True | 0.2644 | 0.934 | True | +0.0 | +0.000 | 24.17 | 24.17 | +0.0 | 0 | 0 |
| 64VPE6 | Castor | 7 | 0.693 | 0.935 | True | 0.693 | 0.935 | True | +0.0 | +0.000 | 2.357 | 2.357 | +0.0 | 0 | 0 |
| 64VPE6 | MTT | 7 | 0.5068 | 0.923 | True | 0.5068 | 0.923 | True | +0.0 | +0.000 | 4.275 | 4.275 | +0.0 | 0 | 0 |
| IsoGE8 | Castor | 7 | 49.61 | -0.080 | False | 49.61 | -0.080 | False | +0.0 | +0.000 | — | — | — | 0 | 0 |
| IsoGE8 | MTT | 7 | 5715 | 0.833 | False | 5715 | 0.833 | False | +0.0 | +0.000 | 93.73 | 93.73 | +0.0 | 0 | 0 |
Series whose fitted IC50 moved >20%: 0 of 8
Series that became reliable: 0; lost reliability: 0
`df` and every section-7 table are unchanged — later sections still use all wells.
# ===========================================================================
# 7g. Trellis: All Antibodies per Linker-Payload, one subplot per Day
# Colour = drug, linestyle/marker = assay (both assays overlaid).
# ===========================================================================
print('\n=== Trellis: dose-response by LP (all antibodies, both assays) ===')
# Build lp -> {ab: drug_name} from all drugs
_lp_ab_map = {}
for d in all_drugs:
ab, lp = _split_ab_lp(d)
if lp is not None:
_lp_ab_map.setdefault(lp, {})[ab] = d
_all_lps = sorted(_lp_ab_map.keys())
_markers = ['o', 's', '^', 'D', 'v', 'P', '*', 'X', 'h', '<']
print(f' Linker-payloads: {_all_lps}')
for lp, abmap in sorted(_lp_ab_map.items()):
print(f' {lp}: {sorted(abmap.keys())}')
# Derive TEST_AB / ISO_AB matched pairs for section 7g (TI trellis)
_test_drugs = {lp: ab_drug[TEST_AB] for lp, ab_drug in _lp_ab_map.items() if TEST_AB in ab_drug}
_iso_drugs = {lp: ab_drug[ISO_AB] for lp, ab_drug in _lp_ab_map.items() if ISO_AB in ab_drug}
_matched_lps = sorted(set(_test_drugs) & set(_iso_drugs))
metric_name = 'Confluence'
pv_col = f'PctViable_{metric_name}'
def _assays_for(drug):
"""Assays this drug was actually measured with, in canonical order."""
return [a for (d, a) in drug_assay_pairs if d == drug]
for lp in _all_lps:
ab_drug = _lp_ab_map[lp]
abs_sorted = sorted(ab_drug.keys())
lp_drugs = [ab_drug[ab] for ab in abs_sorted]
# Pre-scan: y-range within SCALE_MIN_CONC–SCALE_MAX_CONC, excluding SCALE_EXCL_ABS
all_vals = []
for day in fit_days:
for ab, drug in ab_drug.items():
if ab in SCALE_EXCL_ABS:
continue
for assay in _assays_for(drug):
vis = _fit_window(_series_rows(drug, assay, day, pv_col))
if vis.empty:
continue
means = vis.groupby('Conc')[pv_col].mean()
sds = vis.groupby('Conc')[pv_col].std().fillna(0)
all_vals.extend((means + sds).tolist())
all_vals.extend((means - sds).tolist())
if not all_vals:
continue
y_lo = min(all_vals)
y_hi = max(all_vals)
margin = (y_hi - y_lo) * 0.08
ylim = (y_lo - margin, y_hi + margin)
ncols = len(fit_days)
fig, axes = plt.subplots(1, ncols, figsize=(4.6 * ncols, 5), sharey=True)
if ncols == 1:
axes = [axes]
lp_assays = sorted({a for drug in lp_drugs for a in _assays_for(drug)},
key=all_assays.index)
for ax, day in zip(axes, fit_days):
for ab in abs_sorted:
drug = ab_drug[ab]
for assay in _assays_for(drug):
sub = _series_rows(drug, assay, day, pv_col)
if sub.empty:
continue
_plot_series(ax, drug, assay, sub, pv_col, fit_params_pv, day)
ax.set_xscale('log')
ax.set_ylim(ylim)
ax.set_title(f'Day {day}', fontsize=11)
ax.set_xlabel('Conc (nM)')
ax.axhline(50, color='red', ls=':', alpha=0.5)
ax.axhline(100, color='grey', ls=':', alpha=0.3)
if ax is axes[0]:
ax.set_ylabel(minfo['ylabel_pv'])
draw_legend(ax, style_legend_handles(drugs=lp_drugs, assays=lp_assays),
fontsize=7, loc='lower left')
fig.suptitle(f'{lp} — %Viable ({" + ".join(lp_assays)})', fontsize=14, y=1.02)
plt.tight_layout()
save_fig()
=== Trellis: dose-response by LP (all antibodies, both assays) ===
Linker-payloads: []
# ===========================================================================
# 7g. Trellis: Therapeutic Index per concentration, per LP x Day
# TI = %Viable(ISO_AB) / %Viable(each test AB), computed WITHIN an assay.
# ===========================================================================
print('\n=== Trellis: TI by concentration (all test ABs vs ISO_AB, per assay) ===')
metric_name = 'Confluence'
pv_col = f'PctViable_{metric_name}'
_ti_data = {} # (lp, ab, assay, day) -> (conc_array, ti_array)
_lp_yranges = {} # lp -> (ymin, ymax)
for lp in _all_lps:
ab_drug = _lp_ab_map[lp]
if ISO_AB not in ab_drug:
continue
iso_drug = ab_drug[ISO_AB]
test_abs = sorted(ab_drug.keys()) # includes ISO_AB so TI=1 is shown
if not test_abs:
continue
all_ti = []
for ab in test_abs:
drug = ab_drug[ab]
# Only assays where BOTH the test AB and its isotype were measured.
for assay in [a for a in _assays_for(drug) if a in _assays_for(iso_drug)]:
for day in fit_days:
sub_t = _analysis_rows(_series_rows(drug, assay, day, pv_col))
sub_i = _analysis_rows(_series_rows(iso_drug, assay, day, pv_col))
if sub_t.empty or sub_i.empty:
continue
mean_t = sub_t.groupby('Conc')[pv_col].mean()
mean_i = sub_i.groupby('Conc')[pv_col].mean()
shared = sorted(set(mean_t.index) & set(mean_i.index))
if not shared:
continue
concs = np.array(shared)
ti = np.array([mean_i[c] / mean_t[c] if mean_t[c] != 0 else np.nan
for c in shared])
_ti_data[(lp, ab, assay, day)] = (concs, ti)
all_ti.extend(ti[np.isfinite(ti)].tolist())
if all_ti:
_lp_yranges[lp] = (min(all_ti), max(all_ti))
if not _lp_yranges:
print(f' No TI data for {metric_name}')
else:
lps_with_ti = [lp for lp in _all_lps if lp in _lp_yranges]
nrows = len(lps_with_ti)
ncols = len(fit_days)
fig, axes = plt.subplots(nrows, ncols,
figsize=(3.9 * ncols, 3.5 * nrows),
squeeze=False)
for row, lp in enumerate(lps_with_ti):
ab_drug = _lp_ab_map[lp]
test_abs = sorted(ab_drug.keys()) # includes ISO_AB so TI=1 is shown
lp_drugs = [ab_drug[ab] for ab in test_abs]
y_lo, y_hi = _lp_yranges[lp]
margin = (y_hi - y_lo) * 0.1 if y_hi > y_lo else 0.5
ylim = (y_lo - margin, y_hi + margin)
lp_assays = sorted({a for (l, ab, a, d) in _ti_data if l == lp},
key=all_assays.index)
for col_i, day in enumerate(fit_days):
ax = axes[row][col_i]
for ab in test_abs:
drug = ab_drug[ab]
for assay in lp_assays:
key = (lp, ab, assay, day)
if key not in _ti_data:
continue
concs, ti = _ti_data[key]
ax.plot(concs, ti,
marker=assay_markers[assay],
linestyle=assay_linestyles[assay],
color=drug_colors[drug], markersize=5,
linewidth=assay_linewidths[assay],
markeredgecolor='white', markeredgewidth=0.5)
ax.axhline(1, color='red', ls='--', alpha=0.6)
ax.set_xscale('log')
ax.set_ylim(ylim)
draw_legend(ax, style_legend_handles(drugs=lp_drugs, assays=lp_assays),
fontsize=6, loc='best')
if row == 0:
ax.set_title(f'Day {day}', fontsize=11)
if col_i == 0:
ax.set_ylabel(f'{lp}\nTI ({ISO_AB} / AB)', fontsize=9)
else:
ax.set_yticklabels([])
if row == nrows - 1:
ax.set_xlabel('Conc (nM)')
else:
ax.set_xticklabels([])
fig.suptitle(f'Therapeutic Index — %Viable ({metric_name})\n'
f'TI = {ISO_AB} / each test AB, within assay (red dashed = TI 1)',
fontsize=14, y=1.02)
plt.tight_layout()
save_fig()
=== Trellis: TI by concentration (all test ABs vs ISO_AB, per assay) ===
No TI data for Confluence
# ===========================================================================
# 8. Summary tables — side-by-side comparison (plain display version)
# Tables are indexed by Drug x Assay so the two readouts stay distinguishable.
# ===========================================================================
pd.set_option('display.float_format', lambda x: f'{x:.3g}') # applies to all displayed tables
sig3 = lambda x: f'{x:.3g}' # same 3-sig-fig rule, for use inside .style (doesn't inherit the option above)
_ID_COLS = ['Drug', 'Assay']
def _display_pivot(pivot):
"""Return a display-only copy with 'Day' columns prefixed, leaving the original pivot untouched."""
disp = pivot.copy()
disp = disp.rename(columns={c: f'Day {c}' for c in disp.columns if c not in _ID_COLS})
return disp
display(Markdown('## IC50 SUMMARY — COMPARISON'))
for label, df_tbl in all_ic50_tables.items():
if 'interpolated' in label:
continue # handled separately below
# Reliable IC50 pivot
pivot = df_tbl[df_tbl.Reliable].pivot_table(
index=_ID_COLS, columns='Day', values='IC50_nM')
pivot = pivot.rename_axis(columns=None).reset_index()
display(Markdown(f'### {label} — IC50 (nM), reliable only (R²≥{R2_THRESHOLD}, in range)'))
if pivot.empty:
display(Markdown('_(no reliable fits)_'))
else:
display(_display_pivot(pivot).style.hide(axis='index'))
# --- Absolute IC50 tables (direct crossing at 50% of untreated) ---
display(Markdown('## ABSOLUTE IC50 AT 50% OF UNTREATED (no curve fitting)'))
for metric_name in ['Confluence']:
label = f'{metric_name}_interpolated'
df_interp_tbl = all_ic50_tables[label]
pivot = df_interp_tbl.pivot_table(
index=_ID_COLS, columns='Day', values='IC50_interp')
pivot = pivot.rename_axis(columns=None).reset_index()
display(Markdown(f'### {metric_name} — Absolute IC50 (nM) at 50% of untreated'))
display(_display_pivot(pivot).style.hide(axis='index'))
# Side-by-side: absolute 50%-untreated crossing vs fitted midpoint
display(Markdown('### Absolute 50%-untreated vs fitted-midpoint IC50 (per Drug × Assay)'))
interp = all_ic50_tables['Confluence_interpolated'][_ID_COLS + ['Day', 'IC50_interp']]
fitted = all_ic50_tables['Confluence_pctViable']
fitted_rel = fitted[fitted.Reliable][_ID_COLS + ['Day', 'IC50_nM']].rename(
columns={'IC50_nM': 'IC50_LL4'})
compare = interp.merge(fitted_rel, on=_ID_COLS + ['Day'], how='outer')
compare = compare.dropna(subset=['IC50_interp', 'IC50_LL4'], how='all')
compare['Diff_%'] = ((compare.IC50_interp - compare.IC50_LL4) /
compare.IC50_LL4 * 100).round(1)
compare = compare.set_index(_ID_COLS + ['Day']).sort_index()
display(
compare.style
.format({'IC50_interp': sig3, 'IC50_LL4': sig3, 'Diff_%': '{:.1f}'}, na_rep='—')
)
# --- Cross-assay agreement: same drug, one readout against the other ---
if len(all_assays) > 1:
display(Markdown('## CROSS-ASSAY AGREEMENT — fitted IC50 (nM) per drug'))
for tbl_label, value_col, src in [
('Fitted LL.4 midpoint (reliable only)', 'IC50_nM',
all_ic50_tables['Confluence_pctViable'].pipe(lambda t: t[t.Reliable])),
('Absolute 50% of untreated', 'IC50_interp',
all_ic50_tables['Confluence_interpolated']),
]:
x = src.pivot_table(index=['Drug', 'Day'], columns='Assay', values=value_col)
x = x.rename_axis(columns=None)
present = [a for a in all_assays if a in x.columns]
if len(present) == 2:
a0, a1 = present
x[f'{a0}/{a1}'] = x[a0] / x[a1]
display(Markdown(f'### {tbl_label}'))
if x.empty:
display(Markdown('_(no values)_'))
else:
display(x.style.format(sig3, na_rep='—'))| Drug | Assay | Day 7 |
|---|---|---|
| 63GE8 | Castor | 0.050689 |
| 63GE8 | MTT | 0.091955 |
| 64GE6 | Castor | 0.074488 |
| 64GE6 | MTT | 0.207777 |
| 64VPE6 | Castor | 0.613688 |
| 64VPE6 | MTT | 0.492295 |
| IsoGE8 | Castor | 28.833824 |
| Drug | Assay | Day 7 |
|---|---|---|
| 63GE8 | Castor | 0.083133 |
| 63GE8 | MTT | 0.136591 |
| 64GE6 | Castor | 0.091326 |
| 64GE6 | MTT | 0.264357 |
| 64VPE6 | Castor | 0.692950 |
| 64VPE6 | MTT | 0.506750 |
| Drug | Assay | Day 7 |
|---|---|---|
| 63GE8 | MTT | 22.693997 |
| 64GE6 | Castor | 0.856597 |
| 64GE6 | MTT | 24.174933 |
| 64VPE6 | Castor | 2.357239 |
| 64VPE6 | MTT | 4.275096 |
| IsoGE8 | MTT | 93.729343 |
| IC50_interp | IC50_LL4 | Diff_% | |||
|---|---|---|---|---|---|
| Drug | Assay | Day | |||
| 63GE8 | Castor | 7 | — | 0.0831 | — |
| MTT | 7 | 22.7 | 0.137 | 16514.6 | |
| 64GE6 | Castor | 7 | 0.857 | 0.0913 | 838.0 |
| MTT | 7 | 24.2 | 0.264 | 9044.8 | |
| 64VPE6 | Castor | 7 | 2.36 | 0.693 | 240.2 |
| MTT | 7 | 4.28 | 0.507 | 743.6 | |
| IsoGE8 | MTT | 7 | 93.7 | — | — |
| Castor | MTT | Castor/MTT | ||
|---|---|---|---|---|
| Drug | Day | |||
| 63GE8 | 7 | 0.0831 | 0.137 | 0.609 |
| 64GE6 | 7 | 0.0913 | 0.264 | 0.345 |
| 64VPE6 | 7 | 0.693 | 0.507 | 1.37 |
| Castor | MTT | Castor/MTT | ||
|---|---|---|---|---|
| Drug | Day | |||
| 63GE8 | 7 | — | 22.7 | — |
| 64GE6 | 7 | 0.857 | 24.2 | 0.0354 |
| 64VPE6 | 7 | 2.36 | 4.28 | 0.551 |
| IsoGE8 | 7 | — | 93.7 | — |
# ===========================================================================
# 9. IC50 bar plot (reliable, %Viable) — one bar per Drug × Assay
# Bar colour = drug, hatch = assay, matching the curve plots' encoding.
# ===========================================================================
for label, df_tbl in [
('Confluence %Viable', all_ic50_tables['Confluence_pctViable']),
]:
plot_df = df_tbl[df_tbl.Reliable].dropna(subset=['IC50_nM']).copy()
if plot_df.empty:
print(f'{label}: no reliable IC50 values to plot.')
continue
pairs = [(d, a) for (d, a) in drug_assay_pairs
if not plot_df[(plot_df.Drug == d) & (plot_df.Assay == a)].empty]
day_vals = sorted(plot_df.Day.unique())
x = np.arange(len(pairs))
width = 0.8 / max(len(day_vals), 1)
fig, ax = plt.subplots(figsize=(max(8, 1.6 * len(pairs)), 6))
for i, day in enumerate(day_vals):
vals = []
for (d, a) in pairs:
row = plot_df[(plot_df.Drug == d) & (plot_df.Assay == a) &
(plot_df.Day == day)]
vals.append(row.IC50_nM.values[0] if len(row) else np.nan)
bars = ax.bar(x + i * width, vals, width,
color=[drug_colors[d] for (d, _) in pairs],
edgecolor='black', linewidth=0.8,
alpha=1.0 - 0.15 * i)
for bar, (d, a) in zip(bars, pairs):
bar.set_hatch(assay_hatches[a])
ax.set_xticks(x + width * (len(day_vals) - 1) / 2)
ax.set_xticklabels([f'{d}\n{a}' for (d, a) in pairs], fontsize=9)
ax.set_ylabel('IC50 (nM)')
ax.set_yscale('log')
ax.set_title(f'IC50 by Drug × Assay — {label}'
+ (f' (day groups: {", ".join(f"Day {d}" for d in day_vals)})'
if len(day_vals) > 1 else f' (Day {day_vals[0]})'))
handles = [Patch(facecolor=drug_colors[d], edgecolor='black', label=str(d))
for d in sorted({d for (d, _) in pairs})]
handles += [Patch(facecolor='white', edgecolor='black', hatch=assay_hatches[a],
label=str(a)) for a in sorted({a for (_, a) in pairs},
key=all_assays.index)]
ax.legend(handles=handles, fontsize=9, ncol=2)
plt.tight_layout()
save_fig()# ===========================================================================
# 10. ExportDATA_SUBDIR
# ===========================================================================
print('\n=== 10. Export ===')
out_long = os.path.join(DATA_DIR, f'{EXPORT_STEM}_long.csv')
df.to_csv(out_long, index=False)
print(f'Saved long data: {out_long}')
for label, df_tbl in all_ic50_tables.items():
out = os.path.join(DATA_DIR, f'{EXPORT_STEM}_IC50_{label}.csv')
df_tbl.to_csv(out, index=False)
print(f'Saved: {out}')
print(f'\nTotal plots saved: {plot_counter[0]}')
print('Done.')
=== 10. Export ===
Saved long data: /content/drive/MyDrive/20260827 Cytotox - Hu64_VPE6_GE6/Capan Combine Analysis-20260910/Data/Capan Combine Analysis-20260910_long.csv
Saved: /content/drive/MyDrive/20260827 Cytotox - Hu64_VPE6_GE6/Capan Combine Analysis-20260910/Data/Capan Combine Analysis-20260910_IC50_Confluence_interpolated.csv
Saved: /content/drive/MyDrive/20260827 Cytotox - Hu64_VPE6_GE6/Capan Combine Analysis-20260910/Data/Capan Combine Analysis-20260910_IC50_Confluence_direct.csv
Saved: /content/drive/MyDrive/20260827 Cytotox - Hu64_VPE6_GE6/Capan Combine Analysis-20260910/Data/Capan Combine Analysis-20260910_IC50_Confluence_pctViable.csv
Total plots saved: 11
Done.
# ===========================================================================
# 11. Therapeutic Windows — Relative IC50 (midpoint between plateaus)
# Test vs isotype are compared WITHIN an assay; each assay gets its own
# tables and bar charts so Castor and MTT windows are never mixed.
# ===========================================================================
print(f'\n=== 11. Therapeutic Windows ({TEST_AB} vs {ISO_AB}) ===')
print(f' IC50 = relative IC50: conc at midpoint between untreated & max-dose plateaus')
print(f' Assays analysed independently: {all_assays}')
def parse_drug(name, test_ab, iso_ab):
"""Return (antibody, linker-payload) or (None, None) if not matching."""
for ab in [test_ab, iso_ab]:
prefix = ab + '-'
if name.startswith(prefix):
return ab, name[len(prefix):]
return None, None
def relative_ic50(concs, responses):
"""
Relative IC50: concentration where response = midpoint between
the untreated plateau (lowest conc) and the max-dose plateau (highest conc).
Interpolates on log10(conc). Returns (ic50, target_pct, confident).
confident=False if the curve never monotonically crosses the midpoint.
"""
means = responses.groupby(concs).mean()
c = np.array(means.index, dtype=float)
v = np.array(means.values, dtype=float)
order = np.argsort(c)
c, v = c[order], v[order]
untreated = v[0]
max_dose = v[-1]
target = (untreated + max_dose) / 2.0
spread = abs(untreated - max_dose)
if spread < 5.0:
return np.nan, target, False
for i in range(len(v) - 1):
if (v[i] >= target >= v[i+1]) or (v[i] <= target <= v[i+1]):
log_c1, log_c2 = np.log10(max(c[i], 1e-6)), np.log10(max(c[i+1], 1e-6))
frac = (target - v[i]) / (v[i+1] - v[i])
ic50 = 10 ** (log_c1 + frac * (log_c2 - log_c1))
confident = (spread >= 20.0) and (CONC_MIN <= ic50 <= CONC_MAX)
return ic50, target, confident
return np.nan, target, False
METRICS_FOR_TI = {
'Confluence': 'PctViable_Confluence',
}
all_summary_rows = []
for metric_label, pv_col in METRICS_FOR_TI.items():
if pv_col not in df.columns:
print(f'\n {metric_label}: column {pv_col} not found, skipping.')
continue
for assay in all_assays:
assay_df = df[df.Assay == assay]
drugs_in_assay = sorted(assay_df.Drug.dropna().unique())
drug_ab = {d: parse_drug(d, TEST_AB, ISO_AB)[0] for d in drugs_in_assay}
drug_lp = {d: parse_drug(d, TEST_AB, ISO_AB)[1] for d in drugs_in_assay}
test_lps = {drug_lp[d] for d in drugs_in_assay if drug_ab[d] == TEST_AB}
iso_lps = {drug_lp[d] for d in drugs_in_assay if drug_ab[d] == ISO_AB}
matched_lps = sorted(test_lps & iso_lps)
if not matched_lps:
print(f'\n {metric_label} [{assay}]: no matched LPs between '
f'{TEST_AB} and {ISO_AB}.')
continue
days = sorted(assay_df[assay_df.Day.isin(fit_days)].Day.unique())
print(f'\n{"="*60}')
print(f' {metric_label} (%Viable) — assay: {assay}')
print(f' Matched LPs: {matched_lps}')
print(f'{"="*60}')
for day in days:
day_df = assay_df[assay_df.Day == day].copy()
rows = []
for lp in matched_lps:
test_drug = f'{TEST_AB}-{lp}'
iso_drug = f'{ISO_AB}-{lp}'
test_sub = day_df[(day_df.Drug == test_drug)].dropna(subset=['Conc', pv_col])
iso_sub = day_df[(day_df.Drug == iso_drug)].dropna(subset=['Conc', pv_col])
test_sub = _analysis_rows(test_sub)
iso_sub = _analysis_rows(iso_sub)
if test_sub.empty or iso_sub.empty:
continue
t_ic50, t_target, t_conf = relative_ic50(test_sub.Conc, test_sub[pv_col])
i_ic50, i_target, i_conf = relative_ic50(iso_sub.Conc, iso_sub[pv_col])
t_untx = test_sub.groupby('Conc')[pv_col].mean().sort_index().iloc[0]
t_maxd = test_sub.groupby('Conc')[pv_col].mean().sort_index().iloc[-1]
i_untx = iso_sub.groupby('Conc')[pv_col].mean().sort_index().iloc[0]
i_maxd = iso_sub.groupby('Conc')[pv_col].mean().sort_index().iloc[-1]
ti = i_ic50 / t_ic50 if (not np.isnan(t_ic50) and t_ic50 > 0
and not np.isnan(i_ic50)) else np.nan
ti_conf = t_conf and i_conf and not np.isnan(ti)
def fmt(val, conf):
if np.isnan(val):
return 'NA'
return f'{val:.3g}' if conf else f'{val:.3g}*'
rows.append({
'Assay': assay,
'LP': lp,
f'{TEST_AB} untx→max': f'{t_untx:.0f}→{t_maxd:.0f}%',
f'{TEST_AB} IC50 (nM)': fmt(t_ic50, t_conf),
f'{ISO_AB} untx→max': f'{i_untx:.0f}→{i_maxd:.0f}%',
f'{ISO_AB} IC50 (nM)': fmt(i_ic50, i_conf),
f'TI ({ISO_AB}/{TEST_AB})': fmt(ti, ti_conf),
})
all_summary_rows.append({
'Method': metric_label, 'Assay': assay, 'Day': day, 'LP': lp,
f'{TEST_AB}_IC50_nM': t_ic50, f'{TEST_AB}_confident': t_conf,
f'{TEST_AB}_untreated_%': t_untx, f'{TEST_AB}_maxdose_%': t_maxd,
f'{ISO_AB}_IC50_nM': i_ic50, f'{ISO_AB}_confident': i_conf,
f'{ISO_AB}_untreated_%': i_untx, f'{ISO_AB}_maxdose_%': i_maxd,
'TI': ti, 'TI_confident': ti_conf,
})
if not rows:
print(f'\n [{assay}] Day {day}: no matched LP pairs with data.')
continue
tbl = pd.DataFrame(rows)
print(f'\n--- {metric_label} [{assay}] — Day {day} ---')
print(f' IC50 = conc at midpoint of (untreated, max-dose)')
print(f' TI = IC50({ISO_AB}) / IC50({TEST_AB}) ; * = low confidence')
print(tbl.to_string(index=False))
# Grouped bar chart: IC50 by LP
plot_rows = [r for r in rows
if r[f'{TEST_AB} IC50 (nM)'] != 'NA' or r[f'{ISO_AB} IC50 (nM)'] != 'NA']
if not plot_rows:
continue
lps_plot = [r['LP'] for r in plot_rows]
fig, ax = plt.subplots(figsize=(max(6, len(lps_plot) * 2.5), 5))
x = np.arange(len(lps_plot))
width = 0.35
colors = {TEST_AB: '#2196F3', ISO_AB: '#FF9800'}
for j, ab in enumerate([TEST_AB, ISO_AB]):
col_name = f'{ab} IC50 (nM)'
vals, labels, hatches = [], [], []
for r in plot_rows:
raw = r[col_name]
if raw == 'NA':
vals.append(0); labels.append('NA'); hatches.append('//')
else:
v = float(raw.rstrip('*'))
vals.append(v)
labels.append(raw)
hatches.append('//' if raw.endswith('*') else '')
bars = ax.bar(x + (j - 0.5) * width, vals, width, label=ab,
color=colors[ab], alpha=0.85)
for bar, lbl, hatch in zip(bars, labels, hatches):
bar.set_hatch(hatch)
if lbl != 'NA' and bar.get_height() > 0:
ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height(),
lbl, ha='center', va='bottom', fontsize=9)
ax.set_xticks(x)
ax.set_xticklabels(lps_plot, fontsize=11)
ax.set_ylabel('Relative IC50 (nM)')
ax.set_title(f'{metric_label} [{assay}] — Day {day}\n'
f'Relative IC50 by Linker-Payload (* = low confidence)',
fontsize=13)
ax.legend()
ax.set_yscale('log')
plt.tight_layout()
save_fig()
# TI bar chart
ti_rows = [r for r in plot_rows if r[f'TI ({ISO_AB}/{TEST_AB})'] != 'NA']
if ti_rows:
ti_vals = [float(r[f'TI ({ISO_AB}/{TEST_AB})'].rstrip('*')) for r in ti_rows]
ti_lps = [r['LP'] for r in ti_rows]
ti_conf = [not r[f'TI ({ISO_AB}/{TEST_AB})'].endswith('*') for r in ti_rows]
fig, ax = plt.subplots(figsize=(max(5, len(ti_vals) * 2), 4.5))
bar_colors = ['#4CAF50' if c else '#FFC107' for c in ti_conf]
bar_hatches = ['' if c else '//' for c in ti_conf]
bars = ax.bar(range(len(ti_vals)), ti_vals, color=bar_colors, alpha=0.85)
for bar, hatch, v, c in zip(bars, bar_hatches, ti_vals, ti_conf):
bar.set_hatch(hatch)
lbl = f'{v:.1f}' + ('' if c else '*')
ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height(),
lbl, ha='center', va='bottom', fontsize=10, fontweight='bold')
ax.axhline(1.0, color='red', linestyle='--', linewidth=1, label='TI = 1')
ax.set_xticks(range(len(ti_vals)))
ax.set_xticklabels(ti_lps, fontsize=11)
ax.set_ylabel(f'Therapeutic Index\nIC50({ISO_AB}) / IC50({TEST_AB})')
ax.set_title(f'{metric_label} [{assay}] — Day {day}\n'
f'Therapeutic Index (* = low confidence)',
fontsize=13)
ax.legend()
plt.tight_layout()
save_fig()
# Export combined summary
if all_summary_rows:
df_ti_summary = pd.DataFrame(all_summary_rows)
out = os.path.join(DATA_DIR, f'{EXPORT_STEM}_therapeutic_index_{TEST_AB}_vs_{ISO_AB}.csv')
df_ti_summary.to_csv(out, index=False)
print(f'\nSaved combined summary: {out}')
print('\n' + '='*60)
print(' Combined Therapeutic Index Summary (all assays)')
print('='*60)
print(df_ti_summary.to_string(index=False, float_format=lambda x: f'{x:.3g}'))
=== 11. Therapeutic Windows (Hu63 vs Iso) ===
IC50 = relative IC50: conc at midpoint between untreated & max-dose plateaus
Assays analysed independently: ['Castor', 'MTT']
Confluence [Castor]: no matched LPs between Hu63 and Iso.
Confluence [MTT]: no matched LPs between Hu63 and Iso.