Mobile or not mobile, it is a question.

mobile_health_thumb.jpg
Mobile healthcare has been paid a lot of attentions than ever before since 2010. Though mobile devices have been there for a long time, it is believed it is Apple’s iPad that brought people to seriously consider to leverage mobile devices to view medical images. On Feb.4, 2011,FDA cleared first diagnostic radiology application for mobile devices. This event can be regarded as a milestone in healthcare IT industry.

From then on, many image viewing applications have been released to market. These applications can be roughly put into two categories: App based and web-browser based. For those app based, the product takes advantage of supports from underlying operation systems by help of traditional programing languages such as C/C++ , Objective-C or Java to achieve the best performance and native look-and-feel. From user’s perspective, the not-good is the installation is inconvenient. For example, to install an App on iPad,you have to visit the App store via iTunes to download the package. For an average radiologist, this is not always comfortable. In particular for those people from developing countries , it is too complex for them to get started. From developer’s perspective, the downside of native App is the development cost will be rather high once you have to support various operation systems :iOS, Android, Blackberry, Symbian,WebOS ( Thanks God, it has been off the game) and so on so forth.

Though App based products have the apparent disadvantages as mentioned above, it is hard to declare that web-browser-based is better. The user experience is the biggest concern. For now, the html5 based Dicom viewer in mobile devices is far from ideal. There is still some sort of gap to be as good as native App. After all, web based app will run in a browser and heavily rely on the browser’s implementation. In desktop environment, due to the powerful hardware configuration, even the javascript engine is not well optimized, the overall performance is still acceptable. However, once move to a mobile environment, the engine ‘s implementation will overwhelmingly determine the final performance.

There might be some concerns on web browsers’ rendering engine, but it is not true. Nowadays, most of the modern browsers render the context pretty fast. It is never a bottleneck for DICOM viewer. From our experience, the most significant factor which can degrade the performance is array’s implementation of the javascript engine.

HTML5 comes into play like a super star. Its several features make good sense for image processing at least at the first glance. The image pixels can be accessed directly via canvas, the heavy processing work can be delegated to the web workers without interfering the front page, the communication between the web browser and the web server can be established via web socket and so on so forth. However, the performance of JavaScript varies dramatically from browser to browser. It heavily depends on the concrete interpreter implementation. For dicom viewer, the fundamental functionalities include geometrical transformation and gray(color) transformation. For geometrical transformation, interpolation is the core specification. In most of mainstream browsers, the bi-linear algorithm is used but the near neighbor algorithm is not uncommon. It is said from CSS3.0 on, it is allowed for user to switch different algorithms by specifying HTML tag. No matter which algorithm is used, the interpolation speed is very fast because it is believed that the underlying browser engine must have been leveraging the power of multiple cores or GPU if available. From our testing, zoom and pan are not the obstacles when to implement the mobile web viewer. The difficulty mainly results from the gray(color) transformation ,which is normally termed Window/Level in the realm of medical image processing.

Generally speaking, a Window/Level adjustment includes the following steps:
1) Scan the image pixels from the original image array sequentially one by one to read the raw value
2) Find the corresponding item from a built look up table with the raw value as the index. Normally the processing is a random access to the LUT.
3) Write the resulting image array at the same position as at the original, sequentially of course, one by one to build the output image.
In C/C++, the array access, no matter whether it is sequentially read, write or random read, the running time is O(1) so it is very fast. However for Javascript, the underlying implementation of array is not as straightforward as we think. Different browser engine follows different strategy. The O(1) principle doesn’t be reserved any more. From our observations, the O(n) seems in place. We won’t dive into the details but give a simple benchmark here instead.

The test case is design as below: 10 iterations; the input and output image array size 512*512 and 3000*3000. For 16bit depth, the LUT size is 65536, the result is the average time (ms) every iteration takes. The machine is 4 cores Intel i7 2.5GHz with Nvidia Quadro FX 3800M

Fig1.png

Fig2.png

It is obvious that it is unacceptable for big size images to adjust W/L promptly. To overcome such limitation, multiple-threading technique is worth consideration. As we know, Javascript is inherently a single thread language. In HTML5, the web worker is added into the standard to play the similar role as the thread in other languages. Essentially, several web workers may be possibly used in background and each computes parts of image’s window/level simultaneously.

For multiple threading programming, the most difficult part comes from the synchronizing shared data. In Javascript language, a slightly different strategy is used. The front page and the web worker share nothing. All the data which is perhaps shared in other programming languages will be cloned and transferred back and forth between the front page and the web workers. This distributing strategy simplifies the concurrency programming model significantly since now there is no necessity to synchronize anything. However, there are two barriers which prevent the web worker to be used smoothly under current circumstance:
1) The first one is the clone mechanism. For small set of data, it is fine to make a quick clone done, but for image data, the chunk size is usually very big. For instance, in a CR image, the original size is of 3000*3000 or so, suppose we are now working on a four-core computer, we launch four web workers to adjust the window and level simultaneously. For each web worker, we put a workload of 750*750 to do, which means four 750*750 data chunks have to be cloned very quickly from the front page to the web workers for processing and immediately cloned back from the web workers to the front page for displaying. Unfortunately, at least right now, most of web browsers don’t support the clone for big chunk of data. Furthermore, even it is supported, the overhead to clone such big size of data is noticeable. The benefits gained from the concurrency will be totally wasted in a foreseeable way.
2) The second barrier comes from the truth that the html5’s web worker feature is still not widely supported by popular mobile browsers, such as iPad and Android.
webworker.png

In short, it is not possible or practical to take advantage of the web worker to adjust window/level quickly in mobile browser currently. That is to say, for now, Javascript language is not ready to process a big size of image data simultaneously with html5’s web worker mechanism.Fortunately, out product takes a smart strategy to implement the window/level feature otherwise it is impossible to handle the big size of images such as CR/DR. We are very clearly aware that due to javascript engine's limitation, there is a long way to go for the html5 based mobile DICOM viewer to work as well as native App.

Currently, in desktop, from our observation , google chrome is the best one. The good news is that since Android 4.0 , a mobile version of chrome is also available . We have no data on the benchmark yet, but we believe that should work most smoothly among all of the mobile browsers as long as it is using the same engine implementation as the desktop version.

Well, let's talk about our product plan briefly. We are very sure HTML5 based DICOM viewer has a better future than App based. However , this needs some time. It depends on the mobile browsers' evolution. We are very glad to follow with interest the browser war which is happening between those giants like Google, Microsoft, Apple and Mozilla. The intense competition will definitively bring the browsers up to be more powerful. To achieve the same performance as App based app is just a matter of time. Before end of this year, we think there will be a very dramatic improvement for our product's performance in mobile devices.

Even so, we won't wait there passively for the miracle. Beside keeping optimizing our html5 based solution continuously, we are also developing App based native dicom viewer to enrich our product line. Hopefully, the versions for iOS, Android and even Windows 8 will soon be released. Mobile or not mobile , this actually is never a question for us at all!