“Software-Defined Lensing”

Executive Summary

We are proposing to investigate a radical approach to the capture and display of full-spectrum electromagnetic (EM) imaging. This is based on computational principles similar to those exploited in phased array radar systems. Termed “Software-Defined Lensing” (SDL) — our approach relies on coupling two basic innovations:

1)an array of heterodyned, tunable, lightwave nano-antennas; and,

2)a massively parallel rendering engine applying Inverse Fourier Transform (IFT) algorithms to detected EM wavefronts from the nano-array.

What difference would SDL make? An SDL system working in the visible and adjacent IR and UV bands would capture the full information stream contained in light — frequency, phase, polarization — as well as intensity. Conventional photon-counting devices (CCD, CMOS) ignore phase, only capture intensity directly, and require energy-absorbing filtration for broadband frequency discrimination and polarization. Utilizing the full information stream, SDL makes possible detection of target chemical composition, trajectory, dimensions, and features obscured by conventional lensing.

What is the technology behind SDL? By substituting electronics and massively parallel computation for conventional optics we can exploit IFT convolution for image rendering by phase detection, which would ameliorate lens aberrations, augment human perception, and gain significant camera weight and size reduction. We have prototyped an accurate lightwave local oscillator for generating the heterodyne signal.

What is the science behind this technology? Phase information is normally missing from most spectral sensing devices (and all conventional cameras). Wave-based antenna/sensors inherently capture phase information, extrapolating known phased array and synthetic aperture computational analysis for the lightwave EM bands.

Why now? SDL architectures have become feasible due to advances in:

•supercomputer modeling for nano-antenna design and configuration;

•advent of low cost, high performance, multiple input/multiple output parallel processors and adaptive domain-specific signal processing; and,

•non-linear-optic (NLO) frequency mixers for an heterodyned architecture.

What is the military impact if we are successful? Detection and analysis of light in the wave domain — as opposed to simple detection of photons — will lead to revolutionary techniques in imaging for force projection, surveillance, and intelligence:

•lensless, filterless, lightweight, spectral imaging appliances beyond that of conventional multi-spectral cameras for smart weapons and image analysis;

•remote spectral analysis for object recognition, camouflage detection, bio-hazard/ explosive threat detection, harbor and battlefield surveillance;

•field bio-medical applications for (potentially non-invasive) triage; and,

•detecting EM surveillance, augmenting camouflage, confusing attack systems.

Technical Summary

A novel imaging architecture. We propose a seedling effort to research a novel approach to high-resolution electronic imaging that detects electromagnetic (EM) wavefront phases directly, implementing an array of tuned lightwave antennas coupled to a massively parallel rendering engine.

This architecture goes beyond mere target superposition and intensity detection. It combines fundamental knowledge of the wave physics of light with rapidly advancing supercomputer capabilities to facilitate building lensless, lightweight and highly versatile full-spectrum cameras and sensors.

We term this “Software-Defined Lensing” (SDL), as opposed to conventional devices which apply lenses as “mechanical computational devices.” This innovative SDL architecture consists of:

•wave-front sensors implementing a synthetic aperture, phased array of tunable lightwave nano-antennas applying heterodyne de-modulation;

•directly outputting digital data to a multi-nodal, parallel processing, high-performance rendering engine; and,

•applying advanced Inverse Fourier Transform (IFT) algorithms for image reconstruction and downstream spectral and scene analysis.

Such an array would have extraordinary capabilities for extracting the rich data set — phase, frequency and polarization, as well as intensity — contained in light. Normally, this EM data is absent from the best conventional devices.

This architecture exploits intellectual property held by Creative Technology, LLC, (CTech)

How imaging is done now. The mechanism we normally associate with image creation is a lens — a transparent, curved apparatus, which bends light according to the rules of refraction, focusing its rays onto a target, thereby forming a representation in color and intensity of a scene or object. However, what actually takes place in image creation is more complex than simple refraction and ray tracing with its attendant degradations due to lens aberrations.

As Huygens’ Principle illustrates, a glass lens (or curved mirror, etc.) functions as a mechanical “computer” — it delays electromagnetic wavefronts according to “algorithms” embedded in its curvatures, selectively modifying the arrival times of wavelet phases, thereby producing interference patterns on a target varying in intensity and frequency, mapping that of the original scene.

Conventional lenses and light detectors have several critical obstacles for future improvement; likely, only about an order of magnitude at most remains for improving lens and sensor design. Moreover, lenses based on refraction have inherent aberrations and modulation transfer function parameters that distort an image.

Photon counters (eg, CCDs, CMOS) are fundamentally monochromatic, detecting only intensity directly. Lenses and conventional detectors alone cannot detect phase, and require energy-absorbing filtration to segment visible frequencies in broad bands and to detect the various types of polarization. Conventional cameras, due to their relatively broad bandpass filtration, do not even match the full spectral range of the human visual system. More accurate color segmentation, and the ability to detect polarization can be extremely valuable for object spectral signature determination.

What is new in our approach & why we think it will be successful. SDL suggests an alternate method of constructing an image, with the additional attributes of ameliorating lens aberrations and capturing spectral information: direct detection of the phase of wavefronts incident on a target array — without conventional lensing. Instead of using a lens to perform wavefront computation, we propose in this seedling to investigate the potential of extrapolating the large body of synthetic aperture, phased array radar technology from the radiofrequency bands into the light region, utilizing a novel array of lightwave antennas and lightwave heterodyning for de-modulation and accurate frequency (color) tuning.

The SDL concept would enable accurate, point-by-point, temporal knowledge of frequency, phase and wavefront amplitude making it possible to compute narrow bandwidth spectral signatures, including intensity and polarization directly, facilitating downstream processing. “In-focus” narrowband UV and IR would be easier to superimpose on the visible frequencies.

These lightwave analytic parameters are extremely useful beyond mere image creation. Such ultra-spectral signatures can uniquely identify a substance if sufficient information is contained in the light reflected, dispersed, absorbed or refracted by the sample — beyond just its frequency. With conventional filter-based cameras, and most spectrometers, such data acquisition is time-consuming, difficult or impossible based upon their inherent design.

The keys to our lightwave antenna architecture is an heterodyne tuning mechanism, as described in CTech’s US Patent 7,521,680, and implementing the light synthesizer described in CTech’s US Patent 6,985,294 as an accurate local oscillator for the heterodyne mixer/de-modulator. (This light synthesizer has been successfully prototyped, generating any visible frequency in <1 nanometer increments within 0.03 milliseconds, without using energy-absorbing bandpass limiting filters.)

Unlike lightwave detectors that either count photons or simply senses broadband electromagnetic waves, this narrowly tuned antenna architecture extrapolates conventional heterodyne RF technology into the nanometer bands, thus permitting rapid and extremely accurate ultra-spectral detection for downstream computational analysis. Low-noise data output is directly digital, avoiding A/D conversion, thus eliminating sampling impulse and quantizing noise. Antennas, as compared to solid-state photon counters, have inherent gain of several orders of magnitude, hence this architecture incorporates very high sensitivity along with low noise, and little or no requirements for cooling.

How the tuned, heterodyned nano-antennas work. The essence of the lightwave nano-antenna design is a tunable heterodyne receiver/de-modulator mechanism working in the 500 terahertz EM region, coupled to a stable and highly accurate swept frequency local oscillator driving a heterodyne mixer. Since the frequency of the oscillator is known at any specific point in time, sweeping the spectrum yields the phase at each pixel of an array, and hence permits IFT computation and de-convolution to render an image — or for just one pixel, performing chemical analysis.

The heterodyned mechanism exploits properties of non-linear-optic (NLO) frequency mixers working in the optical domain, such as recently discovered (3) NLO materials. One candidate architecture embeds a nano-antenna in the NLO re-radiating substrate, forming the sensor module. There is no physical connection between the sensor module and the down-converter. This method of detection reduces loading effects.

A variable frequency, sweep oscillator (potentially based on the prototype noted above) generates the reference signal for heterodyning. The heterodyne signal emanates from mixing this local oscillator reference with EM energy from the object. CTech’s computer modeling has demonstrated that at the nano-scale required to detect lightwaves a double-spiral antenna’s polar receiving patterns emanate from both the front and back surfaces of the antenna, facilitating the heterodyne mixer configuration. An NLO detector element, integrated with the antenna, enables detecting the heterodyne mixing product received by the non-contact heterodyne detecting antenna.

The reference is swept across a range of frequencies by an electrical pulse delivered from a receiver/processor. When this processor detects a signal it gates an output directly related to the frequency of the object light and simultaneously detects the object’s signal amplitude, therefore uniquely defining the object’s specific spectral characteristics. Phase is derived via temporal computation. Polarization can be detected by a cluster of different antenna configurations. An array of these antenna modules integrated with an SDL, phase-detecting, massively parallel engine enables image rendering via IFT de-convolution.

What is the science behind this innovation? Phase information is normally missing from most spectral sensing devices (and all conventional cameras), yet accurate detection (and control) of phase, along with narrow spectral emission and detection, can be critical for accurate substance analysis.

The wave-based architecture inherently captures accurate phase information by applying known antenna and RF technology to the lightwave region, including the vast body of phased array and synthetic aperture computational analysis.

CTech’s patents cover a tuning mechanism based on known characteristics of heterodyning, dependent on the successful fabrication of an antenna substrate performing as a non-linear mixer in the frequencies of interest. These patents describe a method of wirelessly coupling the substrate to the tuner/modulator minimizing noise and crosstalk, and a local oscillator based on CTech’s prototyped, hyper-accurate and stable light synthesizer.

Why Now? A large body of antenna knowledge is available for supercomputer modeling based on Maxwell’s equations, a task that was difficult or impossible only a few years ago. The concept of “Software-Defined Lensing” is now feasible due to the advent of low cost, high performance, multiple-input multiple-output (MIMO) parallel processing and adaptive domain specific signal processing image engines using CUDA GPU programming techniques.

CTech has modeled a nanostructure antenna configured in lightwave dimensions, applying Maxwell’s equations on a supercomputer. Preliminary results indicate that this design offers vastly improved sensitivity, low noise, and very high dynamic range over a continuous, broad spectral range — from far infrared through the visible to far ultraviolet.

Materials research has progressed to the point whereby various NLO frequency mixers — especially (3) materials — have been developed, and modeling tools are now available. Further, it is now possible to electron beam etch 15 nm two-armed spirals within 1 µm antennas.

Combining the prototyped light-wave synthesizer as a swept frequency local oscillator with a lightwave detecting nano-antenna has a high probably of success for the design and fabrication of a lightwave heterodyne receiver/demodulator.

What are the risks? The nano-antenna array may require experimentation with design and fabrication to optimize performance. At the nano-scale there may be quantum interactions that may be advantageous, or opportunities for creative engineering.

To reduced risk for a full-scale proposal to fabricate and prototype a lightwave phased array architecture, this seedling will focus on acquisition of further critical data for:

1.massively parallel computation for wavefront detection, de-convolution, image rendering and structure analysis, and frequency;

2.evaluating nano-antenna geometries and fabrication for lightwave dimensions (CTech has performed preliminary supercomputer analysis for a nano-antenna);

3.selecting candidate non-linear-optic and similar materials for nano-antenna fabrication and non-linear mixer/detector substrate for an heterodyned signal;

4.alternative light synthesizer local oscillator designs and fabrication, including the interface to the frequency detecting and de-modulator processor.

If our approach is successful, what are the payoffs for the military? Our ultimate goal is an imaging system that rapidly captures full spectral data from a high-resolution spectral scan of a scene —extending the scene’s spectrum into regions invisible to the human eye and beyond that of conventional multi-spectral cameras.

Detection and analysis of light in the wave domain — as opposed to simple detection of photons — will lead to revolutionary techniques in imaging for force projection, surveillance, and intelligence, ie:

•cameras and sensors: lensless, filterless, and lightweight, capable of highly sensitive, accurate spectral imaging over a wide range of narrowly-segmented frequencies, from the IR through the visible to the UV, beyond that of conventional multi-spectral cameras,

•spectral band image registration (UV-VIS-IR) and precision accuracy for smart weapons and image analysis;

•electronic steering and zoom, depth detection without the necessity of stereoscopic triangulation, and occlusion assessment;

•displays: filterless, true-color, full spectrum matched to human visual perception;

•remote spectral analysis for object recognition, camouflage detection, threat detection for bio-hazards and explosives, harbor and battlefield surveillance in difficult visual environments (eg, fog, haze, smoke, night);

•field bio-medical applications for triage and rapid (potentially non-invasive) diagnosis of injuries and bio-hazard exposure;

•hyper-visualization through computation — expanding acutance and spectral data beyond what normal humans can see; and,

•modifying conventional imagers: improving the spatial resolution, and ameliorating aberrations for conventional lensed systems (limited by their modulation transfer properties), by incorporating the output of wave-based sensor architecture with IFT downstream signal processing.

The inverse of these force projection applications is equally important. SDL sensor technology can detect surveillance, augment camouflage, confuse adversary attack systems such as missile guidance, be used for counter-intelligence missions, disinformation, and neutralizing intelligence collection and analysis.

These wave-based sensor technologies will revolutionize imaging capture, display and analysis as we know it today.