Terahertz time-gated spectral imaging for content extraction through layered structures

A. R. Sanchez, B. Heshmat*, A. Aghasi, M. Zhang, S. Naqvi, J. Romberg, R. Raskar



Time of flight imaging has conventionally been used in imaging fast phenomena (Femto photography) and imaging in complex geometries (such as around the corner and through diffusers). Since time and space are tied together with the speed of light (x=ct). Higher time resolutions will result in finer space resolution. In our new study we explore a range of frequencies from 100 Gigahertz to 3 Terahertz (THz) which can penetrate through paper and many other materials. We also sample these pulses of light with very high time resolution (10 trillion frames per second) which is more than 20 times higher than what we used in our previous projects. Such high time resolution provides space resolution of about 30 micron which is high enough to separate pages of a closed book. We have developed an extended algorithm to enable this modality of imaging also known as (THz time domain spectroscopy) to be able to not only image but also read through densely layered structures. This of course has industrial applications in industrial inspection but can also be used to read through closed antique documents or inspect samples of cultural value.

Spatial resolution, spectral contrast, and occlusion are three major bottlenecks in current imaging technologies for non-invasive inspection of complex samples such as closed books. We empower the time-of-flight capabilities of conventional THz time domain spectroscopy and combine it with its spectral capabilities to computationally overcome these bottlenecks. Our study reports successful unsupervised content extraction through a densely layered structure similar to that of a closed book.

Our technique uses time-of-flight-like measurements to localize each page based on the statistics of the reflected THz electric field. After layer localization we use a novel, time-gated spectral analysis that tunes to the highest contrasting frames in spectral domain for each page. Our method provides an order of magnitude enhancement in contrast for the deeper pages. The process finishes off by using a recently introduced method that allows recognition of occluded letters at each page. The study demonstrates the first successful extraction of characters from a stack of at least 9 pages

Enabling THz TDS for extracting content through layered structures is promising due to an abundance of layered structures with sub millimeter layers in industry (e.g. coatings, and polymer-based lamination) and objects of cultural value (e.g. documents, books, and art works).  Therefore, this work represents new capabilities to advance THz and other time-resolved imaging techniques for content extraction through layered structures.
Our demonstration of deep content extraction with THz time-gated imaging opens up several novel directions for imaging and sensing. The work shows that:
1.        Statistics of the electric field in THz TDS can provide much better results for layer localization at deeper layers (lower SNRs) than conventional deconvolution techniques. Furthermore, the technique does not rely on a reference measurement for pulse extraction.


The pages are separated in time using PPEX algorithm even through the pages are pressed together similar to a closed book.


2.       A time-of-flight capability or time resolution that is provided by THz time-domain spectroscopy can be used as a primary tool to spatially resolve the spectral signatures in 3D (rather than being used for solely fourier analysis of the entire sample). This has helped us to tune to small spectral differences between text in and blank paper to high light the text.
3.          Emerging computational methods such as Cardinal Convex Shape decomposition can open up an entire new set of applications by enabling robust recovery of occluded contents.


Spectral contrasting and letter extraction from contrasted images.

Additionally, our technique heavily impacts three fields, namely ultrafast imaging, computational imaging, and Terahertz spectroscopy; we expect that our results will interest a broad audience and provide them with a new platform for inspecting and imaging layered structures.  The work has strong implications for research in industrial imaging, seismic imaging, and cultural heritage. Furthermore, because the imaging modality is in reflection mode, it has potential for extension to long-range imaging applications with other types of time-of-flight sensors.

Access the paper

A. R. Sanchez, B. Heshmat*, A. Aghasi, M. Zhang, S. Naqvi, J. Romberg, R. Raskar, “Terahertz time-gated spectroscopic imaging for content extraction through layered structures”, Nature Communications,

Access the codes and datal

Frequently asked questions

What is ThZ? Terahertz waves or THz waves are electromagnetic waves similar to light that have frequency range of 100 GHz to 10 THz. These frequencies are located in between the far infrared and microwaves and are becoming more and more accessible to the scientists. THz pulses are short burst of light that are made of frequency components from this frequency range.

How does this work relate to the ‘trillion- frames-per-second imaging’ from camera culture group? We use the similar ultrafast imaging methodology to find each page. Based on the measured time of arrival of the THz pulses we calculate the location of each page and then look at the spectrum of the pulse to understand if there is ink on that position of the page or not. The ‘trillion- frames-per-second imaging’ project was the first project that took ultrafast setup commonly used in physics to the world of imaging applications and systems.

Can’t X-ray or ultrasound do this? It may seem that X-ray or ultrasound can also image through a book; however, such techniques lack the contrast of our THz approach for submicron pen or pencil layers compared next to blank paper. These methods have additional drawbacks like cost and ionizing radiation. So while you might be able to hardly detect pages of a closed book if you use a CT scan, you will not be able to see the text. Ultrasound does not have the resolution to detect 20 micron gaps in between the pages of a closed book -distinguishing the ink layers from the blank paper is out of the question for ultrasound. Based on the paper absorption spectrum, we believe that far infrared time resolved systems and THz time domain systems might be the only suitable candidates for investigating paper stacks page by page.
What’s new about this? The core new contributions are 1- A layer extraction method that leverages statistics of the THz signal. This works much better than conventional methods enabling the detection of deeper pages. 2- Spectral contrasting using Kurtosis to tune into the tiny differences in the frequency domain between the plain paper and paper with ink on it. 3- Convex cardinal shape matching that can detect occluded letters. 4- We have used all of these enabling computational methods to exploit both the spectral and time resolution of THz time domain spectroscopy systems to realize a new application.
What are the applications of this technology? THz waves were conventionally used for material study and inspections. Our time-gated spectral content extraction can enable inspection and, more specifically, content extraction from very thin layers much deeper than what was possible before. This has applications in the inspection of samples with cultural value as well as industrial coatings and any properly layered structure. The methodology developed in this paper can also be adapted to other forms of tomographic imaging.
What is the cost of this system? The cost of this system depends on the specifications. For the one used in this study, the cost is around 100k, so it is still not in a position to be mass-produced for everyday use. Today, we expect adoption in specialized industries.
How does this work relate to the ‘optical brush’ or ‘trillion- frames-per-second imaging’ from camera culture group?We use the similar ultrafast imaging methodology to find each page. Based on the measured time of arrival of the THz pulses we calculate the location of each page and then look at the spectrum of the pulse to understand if there is ink on that position of the page or not. The ‘optical brush’ and ‘looking around the corner’ projects also use time of flight to image through an extreme geometry.
What are the limitations of the current first demonstration? The SNR and spectral contrast are still the main bottleneck; a system with higher power can help us read even deeper into the pages. The other limitation is the warping of thin pages. Such warping can create sweeping distortions in the signal, a problem which some of our recent work aims to overcome (see:

Related talks:


Related works:

1.Satat, G. et al. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nat. Commun.6, 67961–67968 (2015).

2.Gariepy, G. et al. Single-photon sensitive light-in-fight imaging. Nat. Commun. 6, 6021 (2015).

3. Kadambi, A. et al. Coded time of flight cameras. ACM Trans. Graph. 32, 1–10 (2013).

4. A. Velten et al., Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).

5.Shen, Y.-C. Terahertz pulsed spectroscopy and imaging for pharmaceutical applications: a review. Int. J. Pharm. 417, 48–60 (2011).

6.Jackson, J. B. et al. A survey of terahertz applications in cultural heritage conservation science. IEEE Trans. Terahertz Sci. Technol. 1, 220–231 (2011).

7.Fukunaga, K. & Hosako, I. Innovative non-invasive analysis techniques for cultural heritage using terahertz technology.Comptes Rendus Phys. 11, 519–526 (2010).

8.Manceau, J.-M., Nevin, A., Fotakis, C. & Tzortzakis, S. Terahertz time domain spectroscopy for the analysis of cultural heritage related materials. Appl. Phys. B 90, 365–368 (2008).

9.J. Takayanagi, et al. “High-resolution time-of-flight terahertz tomography using a femtosecond fiber laser,” Opt. Express 17, 7533-7539 (2009).


The authors would like to thank Prof. Daniel Mittleman at Brown University for discussing the wavelet applications,  David Etayo, Eduardo Azanza, and Alex Lopez at Das Nano, Brian Schulkin and Thomas Tongue at Zomega Terahertz Corporation, and Dr. Harold Hwang and Prof. Keith Nelson at the Massachusetts Institute of Technology for support with experimentation. The authors would also like to thank Ayush Bhandari and Achuta Kadambi for reviewing the pulse extraction concept.


1 2 3
September 13th, 2016

Ramesh Raskar Lemelson-MIT 2016 Winner

Congratulations to Professor Ramesh Raskar and the Camera Culture Group  – Winner of the 2016 Lemelson-MIT prize Ramesh Raskar and […]

September 8th, 2016

Can computers read through a book page by page without opening it?

Terahertz time-gated spectral imaging for content extraction through layered structures A. R. Sanchez, B. Heshmat*, A. Aghasi, M. Zhang, S. […]

November 21st, 2018

Society of Autonomous Vehicles – Part 1

This is part 1 of a multi-part blog series about the Society of Autonomous Vehicles course held in Spring of […]

March 20th, 2018

Seeing Through Realistic Fog

A technique to see through dense, dynamic, and heterogeneous fog conditions. The technique, based on visible light, uses hardware that […]

March 29th, 2017

Efficient Lensless Imaging with a Femto-Pixel

Lensless Imaging with Compressive Ultrafast Sensing Guy Satat, Matthew Tancik, Ramesh Raskar Traditional cameras require a lens and a mega-pixel […]

September 28th, 2016

How to see through tissue

All Photons Imaging Through Volumetric Scattering Guy Satat, Barmak Heshmat, Dan Raviv, Ramesh Raskar We demonstrate a new method that […]

September 12th, 2016

The World is Our Lab

by Roger Archibald Photo credit: John Werner Ramesh Raskar, head of the Media Lab’s Camera Culture Group, takes measure of […]

March 27th, 2016

Handheld 3D Imager to visualize features in the throat like tonsils!

3D visualization of oral cavity and oropharyngeal anatomy may play an important role in the evaluation for obstructive sleep apnea […]

February 24th, 2016

How to use computer vision to improve cities

– Nikhil Naik Transcript from TEDx Beacon Street November, 2015 Here we see a picture of a little girl walking […]

February 12th, 2016

Optical Brush

Optical brush is an open-ended bundle of optical fibers that is enabled with time of flight technology to image and […]

December 3rd, 2015

Innovating for Billions – Ramesh Raskar’s UIST Keynote

28th ACM User Interface Software and Technology Symposium Charlotte, NC November 8-11, 2015 Watch the presentation on Youtube HERE Keynote […]

December 2nd, 2015

Making 3-D imaging 1,000 times better

Algorithms exploiting light’s polarization boost resolution of commercial depth sensors 1,000-fold. See MIT coverage here. Read more about the work […]

November 26th, 2015

Engineering Health Class Final Presentations on 4 Dec

Great projects and demos- wearables, 3D imaging, novel stethoscopes, imaging the eye, oral imaging.  Followed by health night with guest […]

November 2nd, 2015

Engineering Health Class featured in Medtech

Our Engineering Health class was featured in Medtech Boston. Read the full article here.

October 26th, 2015

OPEN POSITIONS: Technical Assistants

Technical Assistant # 1 Date: December-4-2015 The Camera Culture group at the MIT Media Lab focuses on making the invisible […]

October 15th, 2015

Time-of-Flight Microwave Camera to See Through Walls

New Camera Culture research in Nature Scientific Reports shows a prototype time of flight camera working at microwave frequencies. In […]

September 30th, 2015

How to Make a City Smart? The Indian Context

By: Pranav Chandrasekaran   Introduction – by Ramesh Raskar, Associate Professor Media Arts and Sciences, Head of the Camera Culture […]

September 11th, 2015

Kumbh Mela – The World’s Largest Moving City

Kumbh Mela in Nashik Article and Photography by John Werner   “The world’s largest city has no permanent address.”   […]

September 8th, 2015

Engineering Health Fall 2015 Course

MAS.S62: Join us for the first class on Friday, 11 Sept, 1-4 pm in E15-341(Click to go to class webpage)

June 26th, 2015

Vahe Tahmazyan Graduates with Best Thesis