Xamarin Android - CSTORE MPEG4 file problem

[Manos Georgoudakis]

Hi there, 

I'm developing an Android application using version 3.0.2 of the library which takes photos and video files from the Android device, encapsulates them in DICOM files and sends them to an SCP server. This works fine with images. However, when uploading an MPG4 DICOM file I get the familiar 

"codec not registered....."

I know that the Android version of fo dicom only supports a few codecs, mostly for JPG files. Is there any way even by manipulating the MP4 file to upload it to the SCP?

Most importantly, I don't understand why codecs come into play right before the .DCM file to be uploaded. I thought that is just a stream upload and PACS will read all tags to reconstruct the information. Is this accurate?

Kind regards

Manos

[Anders Gustafsson Cureos AB]

Hi Manos,

are you basing your CSTORE implementation on the C-STORE SCP sample? Have you added the various MPEG4 transfer syntaxes to the list of accepted image transfer syntaxes?

Intuitively, I would expect that you should be allowed to store an MPEG4 DICOM file as long as you don't try to render it with fo-dicom. For further investigation, please provide some sample code and a small MP4 DICOM file for testing.

Regards,

Anders @ Cureos

[Manos Georgoudakis]

Thank you for getting back to me so promptly and by the way congratulation for all your effort and excellent work in making these libraries. 

I'm using a ready-made video file from android or tablet (.mp4 files), just convert therm to byte streams and encapsulate them in pixeldata. Here's the full code - 2 functions. 

To be honest I'm not 100% sure that this transfer syntax corresponds to the file format of the produced MP4 files but I had to start from some place. 

  public static string dicomizeVideo(string videoFile, Context ctx)

        {

            

            //videoFile = sessionData._dir.ToString() + "/hale.mpg";

            string dcmOutput = sessionData._dir.CanonicalPath + "/DicomVideoFile_"+ DateTime.Now.ToString("dd.MM.yyyy-HH.mm.ss")+".dcm";

            DateTime currentDate = DateTime.Now;

            string accessionNumber = currentDate.Millisecond.ToString();            

            string patientName = @"Video^Test";

            string patientID = accessionNumber + "9821";

            string patientSex = "F";

            DateTime patientDOB = new DateTime(1990, 1, 15);

            DicomUID studyInstanceUID = DicomUID.AbdominalArteriesLateral12111;

            DicomUID seriesInstanceUID = DicomUID.GeneralRiskFactors6087;

            StringBuilder uid = new StringBuilder();

            uid.Append("1.08.1982.10121984.2.0.07").Append(".z").Append(DateTime.UtcNow.Ticks);

            var sopUID = new DicomUID(uid.ToString(), "SOP Instance UID", DicomUidType.SOPInstance);

            //construct dataset

            DicomDataset dataset = new DicomDataset();

            dataset.Add(DicomTag.SOPClassUID, DicomUID.MultiFrameTrueColorSecondaryCaptureImageStorage);

            dataset.Add(DicomTag.MediaStorageSOPInstanceUID, GenerateUid());

            dataset.Add(DicomTag.StudyInstanceUID, GenerateUid());

            dataset.Add(DicomTag.SeriesInstanceUID, GenerateUid());

            dataset.Add(DicomTag.SOPInstanceUID, GenerateUid());

            dataset.Add(DicomTag.ImageType, "ORIGINAL\\PRIMARY");

            dataset.Add(DicomTag.StudyDate, currentDate);

            dataset.Add(DicomTag.SeriesDate, currentDate);

            dataset.Add(DicomTag.StudyTime, currentDate.ToString());

            dataset.Add(DicomTag.AccessionNumber, accessionNumber);

            dataset.Add(DicomTag.Modality, "OT");

            //dataset.Add(DicomTag.ReferringPhysicianName, referringPhysician);

            dataset.Add(DicomTag.PatientName, patientName);

            dataset.Add(DicomTag.PatientSex, patientSex);

            dataset.Add(DicomTag.PatientBirthDate, patientDOB);

            dataset.Add(DicomTag.PatientID, patientID);            

            dataset.Add(DicomTag.FrameTime, "33.000000");            

            dataset.Add(DicomTag.StudyID, uid.ToString());

            dataset.Add(DicomTag.SeriesNumber, "1238712");

            dataset.Add(DicomTag.SeriesDescription, "MG DICOM Description");

            dataset.Add(DicomTag.InstanceNumber, "2");

            dataset.Add(DicomTag.PhotometricInterpretation, PhotometricInterpretation.YbrPartial420.Value);

            //dataset.Add(DicomTag.PixelAspectRatio, "4\\3");

            dataset.Add(DicomTag.PixelRepresentation, (ushort)0);

            dataset.Add(DicomTag.LossyImageCompression, "01");

            dataset.Add(DicomTag.SpecificCharacterSet, "ISO_IR 100");

            dataset.Add(DicomTag.ConversionType, "SI");

            dataset.Add(DicomTag.Manufacturer, "GlobalMed");

            dataset.Add(DicomTag.BitsAllocated, "8");

            //dataset.Add(DicomTag.TransferSyntaxUID, DicomUID.ImplicitVRLittleEndian);            

            //dataset.Add(DicomTag.PixelData, DicomVR.OB);

            //--if they exist...

            insertUserTags(dataset); 

            //---get video metadata 

            MediaMetadataRetriever mdr = new MediaMetadataRetriever();

            mdr.SetDataSource(videoFile);   

            string vh = mdr.ExtractMetadata(MetadataKey.VideoHeight);

            string vw = mdr.ExtractMetadata(MetadataKey.VideoWidth);

            //string frameRate = mdr.ExtractMetadata(MetadataKey.CaptureFramerate);

            string frameRate = sessionData.videoConfig.frameRate;

            string duration = mdr.ExtractMetadata(MetadataKey.Duration);

            if (frameRate == null) frameRate = "30";

            int frameCount = Convert.ToInt32(frameRate) * Convert.ToInt32(duration)/1000;

            //----add pixeldata info

            DicomPixelData pixelData = DicomPixelData.Create(dataset, true);

            pixelData.Width = Convert.ToUInt16(vw);

            pixelData.Height = Convert.ToUInt16(vh);

            pixelData.NumberOfFrames = frameCount;

            pixelData.HighBit = 7;

            pixelData.BitsStored = 8;

            //pixelData.BitsAllocated = 8;

            pixelData.SamplesPerPixel = 3;

            pixelData.PlanarConfiguration = 0;

            pixelData.PhotometricInterpretation = PhotometricInterpretation.YbrPartial420;

            //---add to tags too (is it necessary??)

            dataset.AddOrUpdate(DicomTag.NumberOfFrames, frameCount);

            dataset.AddOrUpdate(DicomTag.Columns, pixelData.Width);

            dataset.AddOrUpdate(DicomTag.Rows, pixelData.Height);     

            

            //----add video bytes

            byte[] videoBytes = getVideoBytes(videoFile);

            MemoryByteBuffer buffer = new MemoryByteBuffer(videoBytes);

            pixelData.AddFrame(buffer);

            DicomFile dicomfile = new DicomFile(dataset);

            dicomfile.FileMetaInfo.TransferSyntax = DicomTransferSyntax.Lookup(DicomUID.MPEG2MainProfileHighLevel);

            dicomfile.Save(dcmOutput);

            Intent mediaScanIntent = new Intent(Intent.ActionMediaScannerScanFile);    //---update file system        

            return dcmOutput;           

        }

        private static byte[] getVideoBytes(string videoFile)

        {

            FileInputStream fis = new FileInputStream(videoFile);

            ByteArrayOutputStream baos = new ByteArrayOutputStream();

            byte[] buf = new byte[1024];

            int n;

            while (-1 != (n = fis.Read(buf)))

                baos.Write(buf, 0, n);

            byte[] videoBytes = baos.ToByteArray();

            return videoBytes;

        }

In order to send them, I used the SCP example you have on the first page of Git, I have just added some event handling to find out when the transmission ends and if it was successful. Btw, is there any way in 3.0.2 to disable the auto-resend if PACS server closes the connection because it causes endless transmissions?

Here's the send code:

public static async void StoreDicomToPACS(string filepath, Context ctx)

        {

            var client = new DicomClient();

            DicomCStoreRequest sr = new DicomCStoreRequest(filepath);            

            sr.OnResponseReceived = (DicomCStoreRequest rq, DicomCStoreResponse rp) =>

            {

                bool ok = rp.HasDataset;

                bool isStatus = rp.Status.Description == "Success";

                if (isStatus)

                {

                    if (onUploadSuccess != null)

                        onUploadSuccess();

                }

                else

                {

                    if (onUploadFail != null)

                        onUploadFail();

                }

            };

            client.AddRequest(sr);

            try

            {

                await client.SendAsync(sessionData.scp.serverIP, Convert.ToInt16(sessionData.scp.port), sessionData.scp.usesTLS, sessionData.scp.SCU_AE, sessionData.scp.SCP_AE);

            }

            catch (Exception ex)

            {

                AlertDialog.Builder builder1 = new AlertDialog.Builder(ctx);

                builder1.SetMessage("Exception during quering SCU store request. Details: " + ex.Message);

                builder1.SetCancelable(false);

                builder1.SetTitle("Exception during C-Store");

                builder1.SetIcon(Android.Resource.Drawable.IcDialogAlert);

                builder1.SetNeutralButton("OK", (EventHandler<DialogClickEventArgs>)null);

                builder1.Show();

            }

        }

In any case, thank you for your help and support. 

Best regards

Manos

[Manos Georgoudakis]

Correction: the selected transfer syntax (MPEG2 high profile) is apparently wrong, it's left over from some tests. Should be MP4 high profile. Also, the 'sessionData' is a static class of mine which keeps global variables throughout the application. 

[Anders Gustafsson Cureos AB]

OK, so you are using fo-dicom in a C-STORE SCU? Is the SCP configured to receive MP4 DICOM files?

More exactly, where does the application crash, and what is the full exception and error message?

[Manos Georgoudakis]

Yes, the fo-diocm Android application acts as C-STORE SCU. I checked the configuration of the Jivex PACS and noticed that by default the SCP Storage service can accept the  XXX.102 (MPEG4 High profile V41). The application crash the next moment the 

  await client.SendAsync(sessionData.scp.serverIP, Convert.ToInt16(sessionData.scp.port), sessionData.scp.usesTLS, sessionData.scp.SCU_AE, sessionData.scp.SCP_AE);

is called. The exception is:

{Dicom.Imaging.Codec.DicomCodecException: No codec registered for tranfer syntax: MPEG-4 AVC/H.264 High Profile / Level 4.1

  at Dicom.Imaging.Codec.TranscoderManager.GetCodec (Dicom.DicomTransferSyntax syntax) [0x0000f] in C:\fo-dicom\DICOM\Imaging\Codec\TranscoderManager.cs:60 

  at Dicom.Imaging.Codec.DicomTranscoder.get_InputCodec () [0x00015] in C:\fo-dicom\DICOM\Imaging\Codec\DicomTranscoder.cs:53 

  at Dicom.Imaging.Codec.DicomTranscoder.Transcode (Dicom.DicomDataset dataset) [0x000ec] in C:\fo-dicom\DICOM\Imaging\Codec\DicomTranscoder.cs:141 

  at Dicom.Imaging.Codec.DicomCodecExtensions.Clone (Dicom.DicomDataset dataset, Dicom.DicomTransferSyntax syntax, Dicom.Imaging.Codec.DicomCodecParams parameters) [0x0000e] in C:\fo-dicom\DICOM\Imaging\Codec\DicomCodecExtensions.cs:42 

  at Dicom.Network.DicomService.DoSendMessage (Dicom.Network.DicomMessage msg) [0x0041c] in C:\fo-dicom\DICOM\Network\DicomService.cs:992 

  at Dicom.Network.DicomService.SendNextMessage () [0x000c6] in C:\fo-dicom\DICOM\Network\DicomService.cs:866 

  at Dicom.Network.DicomService.SendMessage (Dicom.Network.DicomMessage message) [0x00029] in C:\fo-dicom\DICOM\Network\DicomService.cs:828 

  at Dicom.Network.DicomService.SendRequest (Dicom.Network.DicomRequest request) [0x00000] in C:\fo-dicom\DICOM\Network\DicomService.cs:175 

  at Dicom.Network.DicomClient+<DoSendAsync>d__50.MoveNext () [0x001d2] in C:\fo-dicom\DICOM\Network\DicomClient.cs:411 

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/runtime/exceptionservices/exceptionservicescommon.cs:143 

  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x00047] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/TaskAwaiter.cs:187 

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/TaskAwaiter.cs:156 

  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/TaskAwaiter.cs:128 

  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/TaskAwaiter.cs:113 

  at Medicapture.Controllers.foDICOMController+<StoreDicomToPACS>d__35.MoveNext () [0x000b9] in D:\Dropbox\Projects\Telemed\Medicapture\Medicapture\Controllers\foDICOMController.cs:520 } Dicom.Imaging.Codec.DicomCodecException

On the SCP side, the logged exception is:

26 Apr 2017 07:23:25,511 [DICOM Storage SCP Service - JiveX - 4499 - Executor Thread - 19] WARN DicomStorageSCPService.JiveX - DicomULException : com.visustt.dicomTk.upperLayer.DicomAPAbortException: IOException while reading P-DATA-TF PDU :

com.visustt.dicomTk.upperLayer.DicomAPAbortException: IOException while reading P-DATA-TF PDU : 

    at com.visustt.dicomTk.upperLayer.UpperLayerProtocol.eb(UpperLayerProtocol.java:703)

    at com.visustt.dicomTk.upperLayer.UpperLayerProtocol.ea(UpperLayerProtocol.java:436)

    at com.visustt.dicomTk.dimse.DimseHandler.ea(DimseHandler.java:169)

    at com.visustt.jiveX.services.dicom.DicomStorageSCPHandler.run(DicomStorageSCPHandler.java:460)

    at com.visustt.jiveX.services.general.AbortPooledExecutor$Worker.run(AbortPooledExecutor$Worker.java:837)

    at java.lang.Thread.run(Unknown Source)

  Caused by: java.net.SocketTimeoutException: Read timed out

    at java.net.SocketInputStream.socketRead0(Native Method)

    at java.net.SocketInputStream.socketRead(Unknown Source)

    at java.net.SocketInputStream.read(Unknown Source)

    at java.net.SocketInputStream.read(Unknown Source)

    at java.io.BufferedInputStream.fill(Unknown Source)

    at java.io.BufferedInputStream.read(Unknown Source)

    at java.io.DataInputStream.readByte(Unknown Source)

    at com.visustt.dicomTk.upperLayer.UpperLayerProtocol.eb(UpperLayerProtocol.java:628)

    ... 5 more

It does not make any sense to me why it would call GetCodec just before to send the dicom file. 

[Anders Gustafsson Cureos AB]

Could you please check the value of dataset.InternalTransferSyntax?

I am not sure, but I think you have struck a shortcoming in the fo-dicom library. If the conceived transfer syntax of the dataset is not a transfer syntax that is accepted by the C-STORE SCP, the client attempts to change the transfer syntax to an accepted one, and this change obviously would involve decoding and encoding, hence the attempt to invoke the MPEG4 codec. My suspicion is that the InternalTransferSyntax has not been correctly set in this case, which would explain the codec exception. 

Alternatively, the MPEG4 transfer syntax has not been accepted by the SCP, and you would need to overlook the handshaking so that it is indeed accepted in this particular connection.

If dataset.InternalTransferSyntax is not the desired MPEG4 transfer syntax, I would recommend that you add the pixel data properties manually, i.e. using dataset.Add instead of creating a pixel data object and setting the properties of that object. To set the actual pixel data, use the new DicomDataset.AddOrUpdatePixelData method, where you can specify the transfer syntax of your pixel data directly.

Depending on how things work out for you, I will probably also need to make some improvements in the library, so I would appreciate it if you keep on reporting as extensively as you have done so far. Many thanks!

Regards,

Anders @ Cureos

[Manos Georgoudakis]

Thank you very much, I'll let you know of any results (good or bad)

Kind regards

Manos

[Manos Georgoudakis]

Hi,

Following your advice, I checked in runtime with the debugger the value of the dataset’s internal transfer syntax. Indeed, you were right, the transfer syntax was Explicit VR Little Endian therefore different from the placed transfer syntax as metafile property.

I checked the documentation for AddOrUpdatePixelData regarding the value of DicomVR and noticed that it said that ‘for pixel data should be either OB or OW, else the transfer syntax becomes (surprise – surprise) Explicit VR Little Endian). If you notice my code, you’ll see commented out this:

//dataset.Add(DicomTag.PixelData, DicomVR.OB);

I had seen this in a older example as to how dicomize video but this line caused an exception and so I had it removed.

So I just added this line:

dataset.AddOrUpdatePixelData(DicomVR.OB, buffer, DicomTransferSyntax.Lookup(DicomUID.MPEG4AVCH264HighProfileLevel41));

in this place, at the end:

 

//----add video bytes

            byte[] videoBytes = getVideoBytes(videoFile);

            MemoryByteBuffer buffer = new MemoryByteBuffer(videoBytes);

            pixelData.AddFrame(buffer);           

            dataset.AddOrUpdatePixelData(DicomVR.OB, buffer, DicomTransferSyntax.Lookup(DicomUID.MPEG4AVCH264HighProfileLevel41));

 

            DicomFile dicomfile = new DicomFile(dataset);

            dicomfile.FileMetaInfo.TransferSyntax = DicomTransferSyntax.Lookup(DicomUID.MPEG4AVCH264HighProfileLevel41);           

            dicomfile.Save(dcmOutput);

I still get the same exception.

I’ve seen the configuration file of the JIVEX PACS SPC Storage service and I saw in the list of supported syntaxes the XXX.102 – I’ll be verifying with the company though.

Assuming it’s not a handshake problem (partners of ours have reported successful uploads of MPEG4 dicom files with this particular SCP), can you think of something else?

 

Best regards

M.

 

[Anders Gustafsson Cureos AB]

Thanks, Manos.

I need to give this some additional thought, but right out of the top of my head I sense that we at least need to add a few more built-in transfer syntaxes. Would you be able and willing to share a sample MPEG4 DICOM file for our testing purposes?

Best regards,

Anders

[Manos Georgoudakis]

I'll be sending you the links for 2 files (the .dcm and the respective mp4 file, to your mail. I can also provide you access to the test Jivex installation, should you need to check the handshake process (or just install wireshark to it). 

Thank you for all the help. 

Manos

[Anders Gustafsson Cureos AB]

Hi Manos,

thanks for sending links to the files, I'll download them shortly.

I have moved this entire conversation to Github for the best overview of outstanding development tasks. I would appreciate it if we can continue this discussion on Github from now on.

The link to the Github issue is here.

Regards,

Anders