[Bug] Contour's color is wrong about cornerstone3d HOT 17 OPEN

@alfiehao
Do you mind sharing a code snippet that adds an RTSTRUCT to cornerstone3D's segmentation objects? There were no examples for this, so would like to see your approach.

This is what I did for modality=SEG, but I'd rather load my segmentation data as contours and not a labelmap.

async function fetchAndLoadDCMSeg(searchObj, imageIds){

    // Step 1 - Get search parameters
    const client = new dicomWebClient.api.DICOMwebClient({
        url: searchObj.wadoRsRoot
    });
    const arrayBuffer = await client.retrieveInstance({
        studyInstanceUID: searchObj.StudyInstanceUID,
        seriesInstanceUID: searchObj.SeriesInstanceUID,
        sopInstanceUID: searchObj.SOPInstanceUID
    });

    // Step 2 - Read dicom tags and generate a "toolState".
    // Important keys here are toolState.segmentsOnFrame (for debugging) and toolState.labelmapBufferArray
    const generateToolState =
        await cornerstoneAdapters.adaptersSEG.Cornerstone3D.Segmentation.generateToolState(
            imageIds,
            arrayBuffer,
            cornerstone3D.metaData
        );
    
    // Step 3 - Add a new segmentation to cornerstone3D
    predSegmentationId                = "LOAD_SEGMENTATION_ID:" + cornerstone3D.utilities.uuidv4();
    const {derivedVolume, segReprUID} = await addSegmentationToState(predSegmentationId, cornerstone3DTools.Enums.SegmentationRepresentations.Labelmap);
    const derivedVolumeScalarData     = derivedVolume.getScalarData();
    predSegmentationUIDs              = segReprUID;
    console.log('\n - [fetchAndLoadDCMSeg()] generateToolState: ', generateToolState)

    // Step 4 - Add the dicom buffer to cornerstone3D segmentation 
    derivedVolumeScalarData.set(new Uint8Array(generateToolState.labelmapBufferArray[0]));

}

async function addSegmentationToState(segmentationIdParam, segType){
    // NOTE: segType = cornerstone3DTools.Enums.SegmentationRepresentations.{Labelmap, Contour}

    // Step 0 - Init
    let derivedVolume;

    // Step 1 - Create a segmentation volume
    if (segType === cornerstone3DTools.Enums.SegmentationRepresentations.Labelmap)
        derivedVolume = await cornerstone3D.volumeLoader.createAndCacheDerivedSegmentationVolume(volumeIdCT, {volumeId: segmentationIdParam,});

    // Step 2 - Add the segmentation to the state
    if (segType === cornerstone3DTools.Enums.SegmentationRepresentations.Labelmap)
        cornerstone3DTools.segmentation.addSegmentations([{ segmentationId:segmentationIdParam, representation: { type: segType, data: { volumeId: segmentationIdParam, }, }, },]);
    else if (segType === cornerstone3DTools.Enums.SegmentationRepresentations.Contour)
        cornerstone3DTools.segmentation.addSegmentations([{ segmentationId:segmentationIdParam, representation: { type: segType, }, },]);

    // Step 3 - Set the segmentation representation to the toolGroup
    const segReprUID = await cornerstone3DTools.segmentation.addSegmentationRepresentations(toolGroupId, [
        {segmentationId:segmentationIdParam, type: segType,},
    ]);

    // Step 4 - More stuff for Contour
    if (segType === cornerstone3DTools.Enums.SegmentationRepresentations.Contour){
        const segmentation = cornerstone3DTools.segmentation;
        segmentation.activeSegmentation.setActiveSegmentationRepresentation(toolGroupId,segReprUID[0]);
        segmentation.segmentIndex.setActiveSegmentIndex(segmentationIdParam, 1);
    }
    
    return {derivedVolume, segReprUID}
}

from cornerstone3d.

Which modality are you using for your contours - RTSTRUCT or SEG?

from cornerstone3d.

Which modality are you using for your contours - RTSTRUCT or SEG?

RTSTRUCT

from cornerstone3d.

Weird, we will look

from cornerstone3d.

@prerakmody
I can give my dicom files to you, includ rtstruct file, you can try it.
20971764_jiangyan.zip

And this is my code snippet.

`showSequence(ids){
imageLoader.loadAndCacheImage("wadouri:" + this.rtStructUrl).then(image => {
console.log(image)
}).catch(async (error) => {
if(this.contourSets.length == 0){
let dataSet = error.dataSet;

		let contourData = dataSet.elements['x30060039'].items;
		
		let ROISequence = dataSet.elements['x30060020'].items;
		let ROINumbers = [];
		ROISequence.forEach((item, index) => {
			let ROINumber = item.dataSet.string("x30060022");
			let ROIName = item.dataSet.string("x30060026");
			let color = []
			for(var i=0;i<contourData.length;i++){
				if(contourData[i].dataSet.string('x30060084') == ROINumber){
					color = contourData[i].dataSet.string("x3006002a").split("\\");
					break;
				}
			}
			
			this.ROISequence.push({
				ROINumber,
				ROIName,
				color: color.join(",")
			});
			
			this.ROINumbers.push(ROINumber);
		})
		
		contourData.forEach((item, index) => {
			// console.log(item)
			let color = item.dataSet.string("x3006002a").split("\\");
					
			let sequence = item.dataSet.elements['x30060040'].items;
			// console.log(sequence)
			
			let data = [];
			
			sequence.forEach(s => {
				
				//Referenced SOP Class UID:
				// console.log(s.dataSet.elements['x30060016'].items[0].dataSet.string("x00081150"))
				
				//Referenced SOP Instance UID:
				// console.log(s.dataSet.elements['x30060016'].items[0].dataSet.string("x00081155"))
				
				let ContourGeomrtricType = s.dataSet.string("x30060042");
				
				let ContourPoints = s.dataSet.string("x30060042");
				
				let ContourData = s.dataSet.string("x30060050");
				
				let obj = {
					points: formatPoints(ContourData.split("\\")),
					type: ContourGeomrtricType
				};
				
				data.push(obj);
			})
			
			let number = item.dataSet.string('x30060084');
			
			let contour = {
				data: data,
				id: "contour_" + number,
				color: color,
				number: number,
				segmentIndex: number
			}
			
			contourSets.push(contour);
		})
		
		this.contourSets = contourSets;
	}
	
	await addSegmentationsToState(ids?ids:null);
	
	segmentation.config.setToolGroupSpecificConfig(toolGroupId, {
		renderInactiveSegmentations: true,
		representations: {
			CONTOUR: {
				outlineWidthActive: 1,
				fillAlpha: 0,
			},
		},
	});
	
	//Add the segmentation representation to the toolgroup
	let segRepresentations = await segmentation.addSegmentationRepresentations(toolGroupId, [
		{
			segmentationId,
			type: csToolsEnums.SegmentationRepresentations.Contour,
			options: {
			  polySeg: {
				enabled: true,
			  },
			},
		},
	]);

	planarSegmentationRepresentationUID = segRepresentations[0];
	
	toolGroup.setToolEnabled(SegmentationDisplayTool.toolName);
	
	// Render the image
	renderingEngine.renderViewports([viewportId1, viewportId2, viewportId3]);
	
	this.ROISequenceVisible = true;

})

},`

`async function addSegmentationsToState(ids) {

	segmentation.state.removeSegmentation(segmentationId);
	
	// load the contour data
	const geometryIds = [];
	
	const promises = contourSets.map((contourSet) => {
		
		const geometryId = contourSet.id;
		geometryIds.push(geometryId);
		return geometryLoader.createAndCacheGeometry(geometryId, {
			type: GeometryType.CONTOUR,
			geometryData: contourSet,
		});
	});

	await Promise.all(promises);
	
	// Add the segmentations to state
	segmentation.addSegmentations([
		{
			segmentationId,
			representation: {
				// The type of segmentation
				type: csToolsEnums.SegmentationRepresentations.Contour,
				// The actual segmentation data, in the case of contour geometry
				// this is a reference to the geometry data
				data: {
					geometryIds: ids?ids:geometryIds,
				},
			},
		},
	]);
}

function formatPoints(data){
	let points = [];
	if(data.length == 0){
		return;
	}
	
	for(var i=0; i<data.length / 3; i++){
		let point = data.slice(i * 3, i * 3 + 3)
		points.push([parseFloat(point[0]),parseFloat(point[1]),parseFloat(point[2])]);
	}
	
	return points;
}`

from cornerstone3d.

This screenshot is displayed with OHIF and you can see that the colors are correct.

from cornerstone3d.

Hi,
Thanks for the code and data. I tried visualizing the data through existing tools

1. First, I tried viewer.ohif.org/local - currently OHIF v3.8.3.
   • I was able view the CT and RTSTRUCT in the "Basic Viewer"
     • But as you scroll through the slices, the CT and RTSTRUCT dont align
   • Also, I was unable to view the RTSTRUCT in the "Segmentation" viewer
     • It gave me a generic error: "Unsupported Display Set".
2. I then tried 3D Slicer, but was unable to see both CT or RTSTRUCT.
3. I then simply tried to load the RTSTRUCT file - RS.1.2.752.243.1.1.20240204145607569.8200.20757.dcm into MicroDicom to see the tags, but that failed too.

Also, I was looking to load the data through WADO-RS and not WADO-URI, since my workflow pulls data from an Orthanc Server (with dicom-web extension). But still useful to know how you processed the internal tags of RTSTRUCT to create contourSets and then geometryLoader.createAndCacheGeometry()

from cornerstone3d.

@prerakmody

About your questions:

1、I recorded a video, you can have a look.
2024-07-02 09.30.37.mov.zip

2、Use 3D Slicer, you need to install SlicerRT, you can see my screenshot.

3、I don‘t konw about MicroDicom.

from cornerstone3d.

@alfiehao

• I forgot to install SlicerRT, so thanks for pointing it out.
• If you look at axial slice ids = [72, 77, 78] when uploading to OHIF viewer, you will see some contour misalignments. Weirdly, 3D Slicer shows no such issues.

Check this video

This means that OHIF is somehow not properly reading/rendering your RTSTRUCT. I am also attaching the console logs here, in case @sedghi wants to take a look, since OHIF also uses cornerstone3D.
20971764_jiangyan - viewer.ohif.org-1719922109042.log

from cornerstone3d.

@prerakmody
Hi,
I checked your video. When you want to check RTSTRUCT, you need to load it.

Like this pic.

from cornerstone3d.

@alfiehao

1. That does resolve the issue. UX-wise, seems a bit weird to me that I have to hit "Load". Ideally, a double-click on the RTSTRUCT should simply load it (which is what I did).

2. However, I had no issues loading your data into my own Cornerstone3D (C3D) application using your code snippet above.

   • Currently investigating how to get a RTSTRUCT visible in all 3 views - axial/coronal/sagittal (see image below). Converting to modality=SEG seems like the only way to do this at the moment. Are you looking for something similar?
   • Also, I find that the sliceId displayed in OHIF is incorrect. I think this issue tracker is not the place for OHIF-based discussion, so I have described the issue in community.ohif.org.
     • Please let me know if I can share your data there.

from cornerstone3d.

@prerakmody
You seem to have forgotten my question.
The color was incorrect.

Check this pic.

About modality=SEG, I think it looks bad.
Check this pic.

At last, you can share my data.
Thank you.

from cornerstone3d.

@prerakmody

That does resolve the issue. UX-wise, seems a bit weird to me that I have to hit "Load". Ideally, a double-click on the RTSTRUCT should simply load it (which is what I did).

There are good reasons for this behavior from a user experience perspective. In some situations, there might be 10+ RT files, and users need to know which one to load before proceeding. However, you can remove this behavior by enabling disableConfirmationPrompts in your configuration.

https://docs.ohif.org/configuration/configurationFiles

I believe the issue might be a couple of objects overlaying one another. Have you tried looking at one without the fill?

from cornerstone3d.

@sedghi

There are good reasons for this behavior from a user experience perspective. In some situations, there might be 10+ RT files, and users need to know which one to load before proceeding.

But if the contours are already displayed (with double-click OR drag-and-drop into viewport), then would you not considered them "loaded"? The app has already read the data in the .dcm file. Maybe I am missing something about the "extra loading" when one clicks the "Load" button. Either way, showing misaligned contours in general might not be considered ideal?

Thanks for pointing out the disableConfirmationPrompts option.

I believe the issue might be a couple of objects overlaying one another. Have you tried looking at one without the fill?

Is the "issue" you are referring to here on the misalignment of contours I pointed out in this thread? I only face this issue while using viewer.ohig.org/local and not on my own C3D app. Sorry, but I did not understand "one without the fill".

from cornerstone3d.

@sedghi @prerakmody

Hi,

I think I understand about "one without the fill". Just display one contour without fill, right?
I already tried it, unfortunately, it’s still wrong.

C3D version: 1.81.0

OHIF Viewer

from cornerstone3d.

Is OHIF color-correct? Should it be yellow in the metadata? I'm trying to understand where the issue is occurring. Most likely, it's happening in the metadata provider.

from cornerstone3d.

@sedghi
OHIF is color-correct，C3D v1.23.3 and Slicer are all yellow.

from cornerstone3d.