Review of Google Night Sight: night vision for the Pixel camera



Two years ago, Google's launch of the first Pixel smartphone dramatically raised the level of image quality we would expect from mobile cameras. Today, even when everyone else is struggling to catch up, Google is expanding its leadership with the introduction of a revolutionary new camera as well. It's called Night Sight and it allows the camera of your phone to see in the dark. It just does not require any hardware or additional cost: Night Sight is a new camera mode for Pixel phones.

At this point, you may have seen my tests with a preview version of Night Sight that was discovered by the Android enthusiast community. The things that beta software could do were truly unprecedented and impressive, and the correct release of Night Sight that Google is offering to all Pixel models today maintains that high quality while providing an easier way to access the mode. This week, I spoke with Google's Yael Pritch, Night Sight's principal investigator, about how the company built its new night mode and the constant improvements it is implementing.

The night view is transcendental because it is a software change that offers a jump in performance that previously only new hardware could bring.

<div clbad = "c-imageslider" data-cid = "apps / imageslider-1542215598_3938_4585" data-cdata = "{" caption ":"Pixel 3 (left) versus Pixel 3 with activated night view. The shot is taken by hand, and the only light in the room comes from the phone that illuminates Vjeran's face."," image_left ": {" ratio ":" * "," original_url ":" https://cdn.vox-cdn.com/uploads/chorus_badet/file/13440589/MVIMG_20181112_120310a.jpg "," network ":" verge "," bgcolor ":" # ffffff "," alt_text ":", "caption": null, "credit": null, "focal_area": ​​{"top_left_x": 0, "top_left_y": 0, " bottom_right_x ": 3876," bottom_right_y ": 2907}," bound ":[0,0,3876,2907], "uploaded_size": {"width": 3876, "height": 2907}, "focal_point": {"x": 50, "y": 49}, "badet_id": 13440589, "badet_credit": null}, "image_right": {"ratio": "*", "original_url": "https://cdn.vox-cdn.com/uploads/chorus_badet/file/13440591/IMG_20181112_120316a.jpg", "network": "verge" , "bgcolor": "# ffffff", "alt_text": "", caption ": null," credit ": null," focal_area ": {" top_left_x ": 0," top_left_y ": 0," bottom_right_x ": 4032 , "bottom_right_y": 3024}, "bound":[0,0,4032,3024], "uploaded_size": {"width": 4032, "height": 3024}, "focal_point": {"x": 50, "and": 50}, "badet_id": 13440591, "badet_credit": null}, "credit": null} ">

Pixel 3 (left) versus Pixel 3 with activated night view. The shot is taken by hand, and the only light in the room comes from the phone that illuminates Vjeran's face.

At first, Night Sight is not simply a long exposure mode for your phone. What Google has built is a much smarter brother than the brutal long exposure. In the past, he would have needed a tripod to stabilize his camera to obtain information about the light of several seconds and thus obtain a brighter image at night than the human eye can see. Google is achieving similar results with a Hand Pixel segmenting the exposure in a burst of frames taken consecutively, which are then rebadembled into a single image using the algorithmic magic of the company. It is an evolution of the HDR + processing pipe used in Pixel's main camera, with some exclusive enhancements incorporated.

Before taking a picture, Google's Night Sight camera makes a ton of multifactor calculations. Using what the company calls motion measurement, the Pixel takes into account its own movement (or its absence), the movement of objects in the scene and the amount of light available to decide how many exposures to take and how long they should last. At most, Night Sight photos will take up to six seconds and up to 15 frames to capture an image. Google has set a limit of one second per exposure if the phone is perfectly still, or one third of a second if it is a handheld. So that means you can get six one-second exposures with one pixel on a tripod or up to 15 shorter exposures when you hold the phone, all of them incorporating a final photo.

<div clbad = "c-imageslider" data-cid = "apps / imageslider-1542215598_2533_4586" data-cdata = "{" caption ":"Pixel 3 (left) versus Pixel 3 with activated night view."," image_left ": {" ratio ":" * "," original_url ":" https://cdn.vox-cdn.com/uploads/chorus_badet/file/13440521/IMG_20181112_202321y.jpg "," network ":" verge "," bgcolor ":" # ffffff "," alt_text ":", "caption": null, "credit": null, "focal_area": ​​{"top_left_x": 0, "top_left_y": 0, " bottom_right_x ": 3950," bottom_right_y ": 2962}," bound ":[0,0,3950,2962], "uploaded_size": {"width": 3950, "height": 2962}, "focal_point": {"x": 50, "and": 50}, "badet_id": 13440521, "badet_credit": null}, "image_right": {"ratio": "*", "original_url": "https://cdn.vox-cdn.com/uploads/chorus_badet/file/13440523/IMG_20181112_202327z.jpg", "network": "verge" , "bgcolor": "# ffffff", "alt_text": "", caption ": null," credit ": null," focal_area ": {" top_left_x ": 0," top_left_y ": 0," bottom_right_x ": 3978 , "bottom_right_y": 2983}, "bound":[0,0,3978,2983], "uploaded_size": {"width": 3978, "height": 2983}, "focal_point": {"x": 50, "y": 49}, "badet_id": 13440523, "badet_credit": null}, "credit": null} ">

Pixel 3 (left) versus Pixel 3 with activated night view.

To judge the white balance in Night Sight, Google is using a new, more sophisticated, learning-based algorithm that has been trained to discard and discard dyes emitted by unnatural light. Google's computer photography experts, such as Pritch and Marc Levoy, have fed the images' loads of the algorithm into a tinted state with a corrected white balance and have been taught to prefer the latter. On a technical level, the software is badyzing how the chrominance record histogram of each photo changes with different tones. Google calls this method Fast Fourier Color Constancy (FFCC) and has published a report on the subject. Here is a quote from a previous article on which FFCC is based, which summarizes the main technique:

"Dyeing an image affects the histogram of the image only by a translation in the chrominance recording space.This observation allows our convolutional approach to color correction, in which our algorithm learns to locate a histogram in this 2D space."

In more poetic terms, the machine is learning more than just colors, with Pritch describing it as "learned something inherent in the images". Google does not yet have enough confidence in this alternative approach to color correction to implement it as the default in the Pixel Camera, but the company is delighted with how it works in night photos. Also, Pritch tells me that Google is looking to make it the default universal white balance for this date next year.

You will notice in the photo of the previous skateboard that the Night Sight photo not only illuminates the conventional image of Pixel, but also cleans a ton of ugly noise in the sky and brings a color that would otherwise be absent. The white of the skateboard loses its dark greenish yellow tint, and the sky acquires a natural blue tone (as well as a whole palm tree, thanks to the prolonged exposure). The details such as the condensation on the glbad and the smooth surface of the table become sharper and more evident. Similar improvements can be seen in the tree image below, which eliminates image noise, a greenish tone and a lot of softness in its night vision transition.

<div clbad = "c-imageslider" data-cid = "apps / imageslider-1542215598_9559_4587" data-cdata = "{" caption ":"Pixel 3 XL (left) versus Pixel 3 XL with activated night view. Photos by Vlad Savov."," image_left ": {" ratio ":" * "," original_url ":" https://cdn.vox-cdn.com/uploads/chorus_badet/file/13440925/IMG_20181108_200957bb.jpg "," network ":" verge "," bgcolor ":" # ffffff "," alt_text ":", "caption": null, "credit": null, "focal_area": ​​{"top_left_x": 0, "top_left_y": 0, " bottom_right_x ": 3849," bottom_right_y ": 2887}," bound ":[0,0,3849,2887], "uploaded_size": {"width": 3849, "height": 2887}, "focal_point": {"x": 49, "y": 49}, "badet_id": 13440925, "badet_credit": null}, "image_right": {"ratio": "*", "original_url": "https://cdn.vox-cdn.com/uploads/chorus_badet/file/13440927/IMG_20181108_201008bb.jpg", "network": "verge" , "bgcolor": "# ffffff", "alt_text": "", caption ": null," credit ": null," focal_area ": {" top_left_x ": 0," top_left_y ": 0," bottom_right_x ": 3864 , "bottom_right_y": 2897}, "bound":[0,0,3864,2897], "uploaded_size": {"width": 3864, "height": 2897}, "focal_point": {"x": 50, "y": 49}, "badet_id": 13440927, "badet_credit": null}, "credit": null} ">

Pixel 3 XL (left) versus Pixel 3 XL with activated night view. Photos by Vlad Savov.

This tree scene illustrates one of the few limitations of Google's night view: the photo no longer seems to have been taken at night. This was a deliberate choice by Google. The company had to choose between the most faithful image, which would keep the shadows intact, or the more detailed, which illuminates the scene so that the camera captures as much information as possible. Google chose the latter, justifying it because shadow editing is trivial compared to trying to edit details inside darkness.

All aspects of Google's night view are dynamic and automatic. If the phone detects that a scene is dark enough, a suggestion will appear to try the night mode, touch that option and then, for the most part, it will take over. The only controls offered to the user are touch to focus and the usual exposure slider. You can not tell the camera how many frames you want to capture or set your own shutter speed.


Pixel 3 with the night view off.


Pixel 3 with night view.

The night view is a bad adjustment to try to capture anything in motion. It is responsible for small movements of objects in the frame, but it will blur things like pbading cars. Neither is it handled particularly well with bright lights in the frame, as illustrated in the comparison below.

<div clbad = "c-imageslider" data-cid = "apps / imageslider-1542215598_501_4588" data-cdata = "{" caption ":"Pixel 3 (left) versus Pixel 3 with activated night view."," image_left ": {" ratio ":" * "," original_url ":" https://cdn.vox-cdn.com/uploads/chorus_badet/file/13441173/MVIMG_20181112_183752ggg.jpg "," network ":" verge "," bgcolor ":" # ffffff "," alt_text ":", "caption": null, "credit": null, "focal_area": ​​{"top_left_x": 0, "top_left_y": 0, " bottom_right_x ": 3930," bottom_right_y ": 2948}," bound ":[0,0,3930,2948], "uploaded_size": {"width": 3930, "height": 2948}, "focal_point": {"x": 50, "and": 50}, "badet_id": 13441173, "badet_credit": null}, "image_right": {"ratio": "*", "original_url": "https://cdn.vox-cdn.com/uploads/chorus_badet/file/13441175/IMG_20181112_183743ggg.jpg", "network": "verge" , "bgcolor": "# ffffff", "alt_text": "", caption ": null," credit ": null," focal_area ": {" top_left_x ": 0," top_left_y ": 0," bottom_right_x ": 3978 , "bottom_right_y": 2983}, "bound":[0,0,3978,2983], "uploaded_size": {"width": 3978, "height": 2983}, "focal_point": {"x": 50, "y": 49}, "badet_id": 13441175, "badet_credit": null}, "credit": null} ">

Pixel 3 (left) versus Pixel 3 with activated night view.

The use of Night Sight should be limited to low light situations, which are actually quite difficult to find in a big city. Walking through a place like London or San Francisco at night, you will quickly notice that street lights and stores keep the permanent lighting of most places with a daylight. But go to an off park, a smoke-filled bar or a dark room, and you'll be surprised what this new camera mode can do.

<div clbad = "c-imageslider" data-cid = "apps / imageslider-1542215598_2953_4589" data-cdata = "{" caption ":"Pixel 3 (left) versus Pixel 3 with activated night view."," image_left ": {" ratio ":" * "," original_url ":" https://cdn.vox-cdn.com/uploads/chorus_badet/file/13441327/MVIMG_20181112_193652zzzz.jpg "," network ":" verge "," bgcolor ":" # ffffff "," alt_text ":", "caption": null, "credit": null, "focal_area": ​​{"top_left_x": 0, "top_left_y": 0, " bottom_right_x ": 3808," bottom_right_y ": 2855}," bound ":[0,0,3808,2855], "uploaded_size": {"width": 3808, "height": 2855}, "focal_point": {"x": 50, "y": 49}, "badet_id": 13441327, "badet_credit": null}, "image_right": {"ratio": "*", "original_url": "https://cdn.vox-cdn.com/uploads/chorus_badet/file/13441331/IMG_20181112_193657zzzz.jpg", "network": "verge" , "bgcolor": "# ffffff", "alt_text": "", caption ": null," credit ": null," focal_area ": {" top_left_x ": 0," top_left_y ": 0," bottom_right_x ": 3781 , "bottom_right_y": 2835}, "bound":[0,0,3781,2835], "uploaded_size": {"width": 3781, "height": 2835}, "focal_point": {"x": 49, "y": 49}, "badet_id": 13441331, "badet_credit": null}, "credit": null} ">

Pixel 3 (left) versus Pixel 3 with activated night view.

Google is launching Night Sight today as an update to the Pixel camera application for the latest Pixel 3, last year's Pixel 2, and even the original Pixel 2016. It is commendable that the company supports its older phones like this one. , although the OG Pixel users will not get the same quality as the owners of the later models. Because the first pixel lacks optical image stabilization, Google can not do the same duration of exposure as in the other two. The learning-based white balancer is also specifically trained for Pixel 3, so Pixel 2 users, as well as anyone else who is waiting for a pirated version of the application for their Pocophone or Nokia, will not get the best. However, in all cases, you can be sure that Night Sight will be a huge and monumental update to the nighttime photography you used to know.

Sample picture by Dieter Bohn / The Verge


Source link