top of page
Datamint Logo - White
Search

How centralized data management streamlines medical imaging research

Writer: Rod Fitzsimmons Frey
Rod Fitzsimmons Frey
Sep 9
5 min read

Storing your medical imaging data is a huge and impactful decision that will either bless you or haunt you for the entirety of your research project. In my experience, though, very few researchers set this stuff up intentionally. A pretty typical situation is that they do a PACS export which lands on a hard drive. That fills up or has to move outside the office, so it migrates onto Google Drive or one of its siblings. You realize you have to track which folder each patient is in, so you rename your folders or fire up a spreadsheet.


Then come the medical image segmentations and annotations, so you find a place to put them, maybe update your spreadsheet. More data comes in, you get multiple readers to do annotations on a subset of files, pretty soon your dataset is held together by links and optimism.


This actually works fine for one project, albeit with a bit more cortisol than you might have wanted. But woe betide you if you want to expand your project to another research question, or launch a more formal validation study, or actually reproduce your mid-stream experimental results at some point in the future. And let’s not talk about what happens when your post-doc, Keeper of the Master Laptop, leaves for industry.


What is a dataset, and what does centralized mean?


A medical imaging dataset for an imaging AI study is the scans, the annotations and segmentations, and a set of relationships. Which studies belong to which patient, which annotation was made on which series, by whom, under which version of the annotation protocol. Which cases are in the training set, which are held out for testing, and which were used for a validation pass after the model was frozen.


In a typical lab, the scans live in one place and the relationships live somewhere else. Patient identity sits in the DICOM header until de-identification strips it, then in a key file someone keeps separately; the link between a scan and its segmentation is a folder naming convention; the protocol version usually lives nowhere; the protocol itself is in a Google Doc or in emails to the annotators. Changes are not tracked.


Doing all that management manually is a lot of (annoying) work, but it has consequences for your project, too. Besides the aforementioned traitor of a post-doc who totally promised to be in touch by email. There’s a danger of your spreadsheet getting out of sync with the medical imaging data on the Google Drive. Even worse, in a typical project segmentations and images will continue to arrive and be corrected even as you’re developing the model: without tracking those changes your odds of reproducing intermediate results are approximately zero. And if you continue to evolve the dataset for future research questions, the same reproducibility fail applies to your original (now published!) project.


Centralized medical imaging data management means storing all the associated files and the relationships in the same place as the images, and tracking the changes as time goes on.

Splitting by patient, and being able to show it


If you adopt centralized processes, your skin will clear up, you’ll lose 20 lbs, and your cat will start letting you pet it without clawing you.


Experimental practice will also improve. A good example is the training/test split. If a patient has three studies and you split at the image level, some of that patient's scans land in training and some in test, and your model learns to recognise the patient. Tampu and colleagues measured this on OCT classification: image-level splitting inflated accuracy by 5 to 30 percentage points that were not real.


Most researchers know this, but the difficulty is doing it reliably when patient identity is in a key file and split membership is a folder. When both are attributes of the record, a patient-level split is a filter, and anyone can check it after the fact, including you a year later when a reviewer asks.


Reusing the same scans for the next question


The second gain shows up at project two. A programme's later studies usually draw on the same imaging population as the first. If the first project's scans, de-identification and annotations sit in one indexed place, the second project starts with a filter: this modality, this date range, this tag, cases that already carry this label. If they sit on a drive under a folder scheme built for the first question, it starts with another PACS export, another de-identification pass, and often another conversation with the ethics board about data you already hold.


Annotations carry over, too. A segmentation made for project one is a starting point for project two, provided it is still attached to the scan it was drawn on and to the protocol it was drawn under. A mask file with a matching filename is not quite the same thing if you can’t review the annotation protocol it was create under - do you really remember three years later?


There is a funding angle too. Many agencies, including Canada’s Tri-Agency policy, now require research data management plans at application for certain competitions. A centralized system indicating where the relationships, the files, and the relationships is most of what a credible plan says. With a good system this is boilerplate instead of a torture test of imagination and hope.


Handing the project to the next person


Another gain is continuity. Students rotate on a two to four year cycle but research programmes run longer than that. When the protocol version, the annotator, the review status and the split all live with the data, handing a project over means reading the record. When they do not, it means interviewing the person who left, and they are usually harder to reach than the data. Getting a student productive in days instead of months is good for morale and productivity.


It also connects to something I wrote about annotation quality. When readers adjudicate the cases they disagreed on, the decisions should go back into the protocol. That only helps the next person if the protocol version is attached to the annotations it governed. Otherwise you have a better protocol and no way of knowing which labels were made under the worse one.


Where to start


If you're looking to centralize your medical imaging data, start with one active project, not the archive:


  1. Pick one active project, not the archive.

  2. Decide the unit of record, patient, study or series, and use it everywhere.

  3. Keep the original DICOM untouched. Derive from it and never overwrite it.

  4. Make split membership a stored attribute of each case, decided before annotation starts, so it cannot drift.

  5. Version the annotation protocol and record which version each annotation was made under.

  6. Record who annotated and who reviewed on the record itself, not in a spreadsheet beside it.

  7. Choose a tool that does the above without asking your team to learn a new discipline.


Takeaways


A shared drive and a carefully kept spreadsheet can run a single study perfectly well. Many good papers came out of exactly that, and I am not going to pretend otherwise.


The case for centralizing is reducing your data management workload, enabling the second study, the reviewer's questions about data relationships (especially splits), future reproducibility, and the student who leaves. Each of those is a question about a relationship between pieces of data, and relationships are what a folder cannot hold.


References


 
 
 

Comments

bottom of page