Terahertz time-gated spectral imaging for content extraction through layered structures

A. R. Sanchez, B. Heshmat*, A. Aghasi, M. Zhang, S. Naqvi, J. Romberg, R. Raskar

Abstract

Time of flight imaging has conventionally been used in imaging fast phenomena (Femto photography) and imaging in complex geometries (such as around the corner and through diffusers). Since time and space are tied together with the speed of light (x=ct). Higher time resolutions will result in finer space resolution. In our new study we explore a range of frequencies from 100 Gigahertz to 3 Terahertz (THz) which can penetrate through paper and many other materials. We also sample these pulses of light with very high time resolution (10 trillion frames per second) which is more than 20 times higher than what we used in our previous projects. Such high time resolution provides space resolution of about 30 micron which is high enough to separate pages of a closed book. We have developed an extended algorithm to enable this modality of imaging also known as (THz time domain spectroscopy) to be able to not only image but also read through densely layered structures. This of course has industrial applications in industrial inspection but can also be used to read through closed antique documents or inspect samples of cultural value.

Spatial resolution, spectral contrast, and occlusion are three major bottlenecks in current imaging technologies for non-invasive inspection of complex samples such as closed books. We empower the time-of-flight capabilities of conventional THz time domain spectroscopy and combine it with its spectral capabilities to computationally overcome these bottlenecks. Our study reports successful unsupervised content extraction through a densely layered structure similar to that of a closed book.

Our technique uses time-of-flight-like measurements to localize each page based on the statistics of the reflected THz electric field. After layer localization we use a novel, time-gated spectral analysis that tunes to the highest contrasting frames in spectral domain for each page. Our method provides an order of magnitude enhancement in contrast for the deeper pages. The process finishes off by using a recently introduced method that allows recognition of occluded letters at each page. The study demonstrates the first successful extraction of characters from a stack of at least 9 pages

Enabling THz TDS for extracting content through layered structures is promising due to an abundance of layered structures with sub millimeter layers in industry (e.g. coatings, and polymer-based lamination) and objects of cultural value (e.g. documents, books, and art works).  Therefore, this work represents new capabilities to advance THz and other time-resolved imaging techniques for content extraction through layered structures.
Our demonstration of deep content extraction with THz time-gated imaging opens up several novel directions for imaging and sensing. The work shows that:
1.        Statistics of the electric field in THz TDS can provide much better results for layer localization at deeper layers (lower SNRs) than conventional deconvolution techniques. Furthermore, the technique does not rely on a reference measurement for pulse extraction.

The pages are separated in time using PPEX algorithm even through the pages are pressed together similar to a closed book.

2.       A time-of-flight capability or time resolution that is provided by THz time-domain spectroscopy can be used as a primary tool to spatially resolve the spectral signatures in 3D (rather than being used for solely fourier analysis of the entire sample). This has helped us to tune to small spectral differences between text in and blank paper to high light the text.
3.          Emerging computational methods such as Cardinal Convex Shape decomposition can open up an entire new set of applications by enabling robust recovery of occluded contents.

Spectral contrasting and letter extraction from contrasted images.

Additionally, our technique heavily impacts three fields, namely ultrafast imaging, computational imaging, and Terahertz spectroscopy; we expect that our results will interest a broad audience and provide them with a new platform for inspecting and imaging layered structures.  The work has strong implications for research in industrial imaging, seismic imaging, and cultural heritage. Furthermore, because the imaging modality is in reflection mode, it has potential for extension to long-range imaging applications with other types of time-of-flight sensors.

Access the paper

A. R. Sanchez, B. Heshmat*, A. Aghasi, M. Zhang, S. Naqvi, J. Romberg, R. Raskar, “Terahertz time-gated spectroscopic imaging for content extraction through layered structures”, Nature Communications, www.nature.com/articles/srep20217

Access the codes and data

https://figshare.com/articles/CCSC_kurtosis_and_wavelet_codes_and_results_zip/3471434/1

Frequently asked questions

What is ThZ? Terahertz waves or THz waves are electromagnetic waves similar to light that have frequency range of 100 GHz to 10 THz. These frequencies are located in between the far infrared and microwaves and are becoming more and more accessible to the scientists. THz pulses are short burst of light that are made of frequency components from this frequency range.

How does this work relate to the ‘trillion- frames-per-second imaging’ from camera culture group? We use the similar ultrafast imaging methodology to find each page. Based on the measured time of arrival of the THz pulses we calculate the location of each page and then look at the spectrum of the pulse to understand if there is ink on that position of the page or not. The ‘trillion- frames-per-second imaging’ project was the first project that took ultrafast setup commonly used in physics to the world of imaging applications and systems.

Can’t X-ray or ultrasound do this? It may seem that X-ray or ultrasound can also image through a book; however, such techniques lack the contrast of our THz approach for submicron pen or pencil layers compared next to blank paper. These methods have additional drawbacks like cost and ionizing radiation. So while you might be able to hardly detect pages of a closed book if you use a CT scan, you will not be able to see the text. Ultrasound does not have the resolution to detect 20 micron gaps in between the pages of a closed book -distinguishing the ink layers from the blank paper is out of the question for ultrasound. Based on the paper absorption spectrum, we believe that far infrared time resolved systems and THz time domain systems might be the only suitable candidates for investigating paper stacks page by page.
What’s new about this? The core new contributions are 1- A layer extraction method that leverages statistics of the THz signal. This works much better than conventional methods enabling the detection of deeper pages. 2- Spectral contrasting using Kurtosis to tune into the tiny differences in the frequency domain between the plain paper and paper with ink on it. 3- Convex cardinal shape matching that can detect occluded letters. 4- We have used all of these enabling computational methods to exploit both the spectral and time resolution of THz time domain spectroscopy systems to realize a new application.
What are the applications of this technology? THz waves were conventionally used for material study and inspections. Our time-gated spectral content extraction can enable inspection and, more specifically, content extraction from very thin layers much deeper than what was possible before. This has applications in the inspection of samples with cultural value as well as industrial coatings and any properly layered structure. The methodology developed in this paper can also be adapted to other forms of tomographic imaging.
What is the cost of this system? The cost of this system depends on the specifications. For the one used in this study, the cost is around 100k, so it is still not in a position to be mass-produced for everyday use. Today, we expect adoption in specialized industries.
How does this work relate to the ‘optical brush’ or ‘trillion- frames-per-second imaging’ from camera culture group?We use the similar ultrafast imaging methodology to find each page. Based on the measured time of arrival of the THz pulses we calculate the location of each page and then look at the spectrum of the pulse to understand if there is ink on that position of the page or not. The ‘optical brush’ and ‘looking around the corner’ projects also use time of flight to image through an extreme geometry.
What are the limitations of the current first demonstration? The SNR and spectral contrast are still the main bottleneck; a system with higher power can help us read even deeper into the pages. The other limitation is the warping of thin pages. Such warping can create sweeping distortions in the signal, a problem which some of our recent work aims to overcome (see:https://www.osapublishing.org/optica/abstract.cfm?URI=optica-3-7-754).

Related talks:

Related works:

1.Satat, G. et al. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nat. Commun.6, 67961–67968 (2015).

2.Gariepy, G. et al. Single-photon sensitive light-in-fight imaging. Nat. Commun. 6, 6021 (2015).

3. Kadambi, A. et al. Coded time of flight cameras. ACM Trans. Graph. 32, 1–10 (2013).

4. A. Velten et al., Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).

5.Shen, Y.-C. Terahertz pulsed spectroscopy and imaging for pharmaceutical applications: a review. Int. J. Pharm. 417, 48–60 (2011).

6.Jackson, J. B. et al. A survey of terahertz applications in cultural heritage conservation science. IEEE Trans. Terahertz Sci. Technol. 1, 220–231 (2011).

7.Fukunaga, K. & Hosako, I. Innovative non-invasive analysis techniques for cultural heritage using terahertz technology.Comptes Rendus Phys. 11, 519–526 (2010).

8.Manceau, J.-M., Nevin, A., Fotakis, C. & Tzortzakis, S. Terahertz time domain spectroscopy for the analysis of cultural heritage related materials. Appl. Phys. B 90, 365–368 (2008).

9.J. Takayanagi, et al. “High-resolution time-of-flight terahertz tomography using a femtosecond fiber laser,” Opt. Express 17, 7533-7539 (2009).

Acknowledgements

The authors would like to thank Prof. Daniel Mittleman at Brown University for discussing the wavelet applications,  David Etayo, Eduardo Azanza, and Alex Lopez at Das Nano, Brian Schulkin and Thomas Tongue at Zomega Terahertz Corporation, and Dr. Harold Hwang and Prof. Keith Nelson at the Massachusetts Institute of Technology for support with experimentation. The authors would also like to thank Ayush Bhandari and Achuta Kadambi for reviewing the pulse extraction concept.