Anyone who has shipped echo segmentation into a product knows the model zoo problem. The apical 4-chamber model does not work on a parasternal short-axis clip, the 2-chamber model has never seen a right ventricle, and every view gets its own weights, its own validation and its own bug reports. A team from the University of British Columbia, with co-authors at the University of Regensburg, EPFL and Rutgers, tries to collapse that into one network. Their model, P2Echo (Prompt2Echo), takes an echo frame plus a short text prompt that names the view and the structure, and returns the mask for that structure. The paper is arXiv:2611.13425, and the code is public under the MIT license.
What P2Echo does
P2Echo covers four standard views (apical 2-, 3- and 4-chamber, and parasternal short axis) and five labels: left ventricle (LV), myocardium (MYO), left atrium (LA), right ventricle (RV) and right atrium (RA). Instead of loading a model per view, you pass a prompt.
The prompt follows a fixed template: the view, the target, and a one-line anatomical description of what that target looks like in that view. For the LV in an apical 4-chamber view, the description says it is bullet shaped and forms the apex. In short axis, the LV is circular at mid ventricle and the RV is a crescent around it. The authors say the templates were written with radiologist input. A frozen Qwen3-Embedding-0.6B model turns the prompt into an embedding. Because the set of prompts is fixed, those embeddings can be computed once and cached, so the language model adds almost nothing at inference.
The image side is a PVTv2-B2 encoder, an ImageNet-pretrained vision transformer. A small cross-attention decoder grounds the prompt in the deepest image features, then the result is injected into every decoder stage through a block the authors call DyITA (Dynamic Image-Text Alignment). Inside it, small routers pick one of several learned projections per token and subtract a “redundant” component before attention, so the decoder features get re-weighted at every resolution according to what the prompt asked for. In the ablation, removing any of the three DyITA parts costs between 1.70 and 2.19 Dice points on the hardest external set.
The output is one probability map per prompt. Ask for LV and LA in a 4-chamber frame and you get two maps, and an argmax over them gives the label image. Inputs are single 2D frames at 256 × 256 pixels.
What they found
The training pool is five public datasets harmonized into one label space: CAMUS, CardiacUDA, EchoCP, EchoNet-Pediatric and SegRWMA. Each one labels different structures in different views, and the authors had to remap pixel values, rasterize contours and, for one dataset, fill the LV cavity from a myocardium-only outline. Three more datasets were kept out of training entirely for external testing: HMC-QU, EchoNet-Dynamic and CardiacNet.
On the internal test split P2Echo beats nnU-Net, SAM-based models and EchoONE, the previous multi-view echo model, on 2CH, 3CH and 4CH. The biggest gap is 2CH, with a mean Dice of 96.94 against 92.82 for nnU-Net and 91.69 for EchoONE. On PSAX it is level with the others (92.00 against 91.71 for EchoONE).
On HMC-QU, a low-quality 4-chamber set collected for myocardial infarction detection, P2Echo reaches 82.22 mean Dice while nnU-Net drops to 45.30 and EchoONE to 53.96. The authors point to the training data. Only CAMUS labels myocardium, and models trained on mixed datasets tend to learn that “unlabeled” means “absent”. On EchoNet-Dynamic and CardiacNet the margins are a point or two, and on EchoNet-Dynamic CENet has a slightly better HD95.
Off-the-shelf text-prompted foundation models did badly. Zero-shot SAM3 scored under 10 Dice on every internal view, and MedCLIP-SAMv2 stayed under 32. Fine-tuned SAM3 was competitive on accuracy but took 0.082 s per image against 0.0155 s for P2Echo on an H100 (64.68 frames per second, 6.90 GFLOPs, 37.47M parameters).
Why a viewer or clinic team should care
The first gain is maintenance. One set of weights, one preprocessing path and one regression suite for four views is cheaper to run than four specialists, and the paper shows you do not pay for it in accuracy on these datasets.
The prompt is also a clean API. In a viewer, the view label already comes from the acquisition, a view classifier or the sonographer. You map “4CH” plus the structures you want to a cached embedding and call the model. When nothing sensible is asked for, P2Echo returns close to nothing: an ambiguous prompt (“Segment the cardiac chamber”) scored 0.01 Dice, a null prompt 0.00, and “Segment the liver” 6.35. For a product, empty output on a bad request is easier to handle than a confident wrong mask.
At about 15 ms per frame on an H100, it is fast enough to run over every frame of a cine loop for review. That speed is measured on a data-center GPU, though, not on a cart-based scanner.
What it does not tell you yet
- The prompt is brittle. On 450 internal images, the trained template scored 93.53 Dice. A clinical paraphrase (“Delineate the left ventricular blood pool”) dropped to 80.48, and a short prompt (“Segment the left ventricle cavity”) to 35.83 with a standard deviation of 36. Do not expose a free-text box to users. Hard-code the templates.
- The view must be known. The model assumes you tell it the view, and it was only tested on four. Wrong view in the prompt is not tested.
- Thin supervision for 3CH and PSAX. Each comes from a single development dataset, and both of those label only the LV. The PSAX data is pediatric (EchoNet-Pediatric). The RV crescent in PSAX that the prompt table describes has no ground truth in training.
- Single frames. There is no temporal model, so frame-to-frame consistency across a cine loop is not measured.
- No clinical measurements. The paper reports Dice, IoU and HD95 only. Ejection fraction, volumes and strain are left to future work.
- Beyond-annotation masks are unscored. Figure 5 shows plausible masks for structures that have no label in the source data, but there is nothing to score them against.
How to try it
The repository moeinheidari7829/P2Echo has training, test and benchmark scripts under an MIT license, and the README links a trained checkpoint on Google Drive. The datasets are not redistributed. You download each from its official source and convert frames and labels to PNG with the unified label map (0 background, 1 LV, 2 MYO, 3 LA, 4 RV, 5 RA). The checkpoint was trained on those public datasets, each with its own terms of use, so read them before putting the weights in a commercial product. The README still has a placeholder where the arXiv id goes, and the GitHub description cites a different arXiv number. The paper is 2611.13425.
A sensible first test is your own 4-chamber clips with the canonical prompts, scored against a few hand-drawn masks, before trusting it on 3CH or short axis. For related echo work on the blog, see ejection fraction from the parasternal long axis and EchoAppearance, which rebuilds the vendor image look from raw beam data.
Source and license
Paper: Moein Heidari, Alvin Kimbowa, Mahdi Tabatabaei, Afshin Bozorgpour, Sina Ghorbani Kolahi, River Jiang, Wenjin Chen, David J. Foran, Jasmine Grewal, Dorit Merhof and Ilker Hacihaliloglu, “Prompt2Echo (P2Echo): Text-Prompted Multi-View Echocardiography Segmentation with One Model”, arXiv:2611.13425 (cs.CV, 10 November 2026). Affiliations: The University of British Columbia (School of Biomedical Engineering, Department of Radiology, Division of Cardiology), University of Regensburg, EPFL and Rutgers University. The arXiv version is a manuscript submitted to an IEEE ultrasonics journal on 12 September 2026, not yet a published article. Code: github.com/moeinheidari7829/P2Echo, MIT license, checkpoint linked from the README. Featured image: Fig. 4 of the paper, four of nine columns, panels cropped and scaled.