How Instagram enabled users to take stunning Low Light Photos

Posted by Donovan McMurray – Developer Relations Engineer

Instagram, the popular photo and video sharing social networking service, is constantly delighting users with a best-in-class camera experience. Recently, Instagram launched another improvement on Android with their Night Mode implementation.

As devices and their cameras become more and more capable, users expect better quality images in a wider variety of settings. Whether it’s a night out with friends or the calmness right after you get your baby to fall asleep, the special moments users want to capture often don’t have ideal lighting conditions.

Now, when Instagram users on Android take a photo in low light environments, they’ll see a moon icon that allows them to activate Night Mode for better image quality. This feature is currently available to users with any Pixel device from the 6 series and up, a Samsung Galaxy S24Ultra, or a Samsung Flip6 or Fold6, with more devices to follow.

Leveraging Device-specific Camera Technologies

Android enables apps to take advantage of device-specific camera features through the Camera Extensions API. The Extensions framework currently provides functionality like Night Mode for low-light image captures, Bokeh for applying portrait-style background blur, and Face Retouch for beauty filters. All of these features are implemented by the Original Equipment Manufacturers (OEMs) in order to maximize the quality of each feature on the hardware it’s running on.

Furthermore, exposing this OEM-specific functionality through the Extensions API allows developers to use a consistent implementation across all of these devices, getting the best of both worlds: implementations that are tuned to a wide-range of devices with a unified API surface. According to Nilesh Patel, a Software Engineer at Instagram, “for Meta’s billions of users, having to write custom code for each new device is simply not scalable. It would also add unnecessary app size when Meta users download the app. Hence our guideline is ‘write once to scale to billions’, favoring platform APIs.”

More and more OEMs are supporting Extensions, too! There are already over 120 different devices that support the Camera Extensions, representing over 75 million monthly active users. There’s never been a better time to integrate Extensions into your Android app to give your users the best possible camera experience.

Impact on Instagram

The results of adding Night Mode to Instagram have been very positive for Instagram users. Jin Cui, a Partner Engineer on Instagram, said “Night Mode has increased the number of photos captured and shared with the Instagram camera, since the quality of the photos are now visibly better in low-light scenes.”

Compare the following photos to see just how big of a difference Night Mode makes. The first photo is taken in Instagram with Night Mode off, the second photo is taken in Instagram with Night Mode on, and the third photo is taken with the native camera app with the device’s own low-light processing enabled.

Ensuring Quality through Image Test Suite (ITS)

The Android Camera Image Test Suite (ITS) is a framework for testing images from Android cameras. ITS tests configure the camera and capture shots to verify expected image data. These tests are functional and ensure advertised camera features work as expected. A tablet mounted on one side of the ITS box displays the test chart. The device under test is mounted on the opposite side of the ITS box.

Devices must pass the ITS tests for any feature that the device claims to support for apps to use, including the tests we have for the Night Mode Camera Extension.

The Android Camera team faced the challenge of ensuring the Night Mode Camera Extension feature functioned consistently across all devices in a scalable way. This required creating a testing environment with very low light and a wide dynamic range. This configuration was necessary to simulate real-world lighting scenarios, such as a city at night with varying levels of brightness and shadow, or the atmospheric lighting of a restaurant.

The first step to designing the test was to define the specific lighting conditions to simulate. Field testing with a light meter in various locations and lighting conditions was conducted to determine the target lux level. The goal was to ensure the camera could capture clear images in low-light conditions, which led to the establishment of 3 lux as the target lux level. The figure below shows various lighting conditions and their respective lux value.

The next step was to develop a test chart to accurately measure a wide dynamic range in a low light environment. The team developed and iterated on several test charts and arrived at the following test chart shown below. This chart arranges a grid of squares in varying shades of grey. A red outline defines the test area for cropping. This enables excluding darker external regions. The grid follows a Hilbert curve pattern to minimize abrupt light or dark transitions. The design allows for both quantitative measurements and simulation of a broad range of light conditions.

The test chart captures an image using the Night Mode Camera Extension in low light conditions. The image is used to evaluate the improvement in the shadows and midtones while ensuring the highlights aren’t saturated. This evaluation involves two criteria: ensure the average luma value of the six darkest boxes is at least 85, and ensure the average luma contrast between these boxes is at least 17. The figure below shows the test capture and chart results.

By leveraging the existing ITS infrastructure, the Android Camera team was able to provide consistent, high quality Night Mode Camera Extension captures. This gives application developers the confidence to integrate and enable Night Mode captures for their users. It also allows OEMs to validate their implementations and ensure users get the best quality capture.

How to Implement Night Mode with Camera Extensions

Camera Extensions are available to apps built with Camera2 or CameraX. In this section, we’ll walk through each of the features Instagram implemented. The code examples will use CameraX, but you’ll find links to the Camera2 documentation at each step.

Enabling Night Mode Extension

Night Mode involves combining multiple exposures into a single still photo for better quality shots in low-light environments. So first, you’ll need to check for Night Mode availability, and tell the camera system to start a Camera Extension session. With CameraX, this is done with an ExtensionsManager instead of the standard CameraManager.

private suspend fun setUpCamera() {
  // Obtain an instance of a process camera provider. The camera provider
  // provides access to the set of cameras associated with the device.
  // The camera obtained from the provider will be bound to the activity lifecycle.
  val cameraProvider = ProcessCameraProvider.getInstance(application).await()

  // Obtain an instance of the extensions manager. The extensions manager 
  // enables a camera to use extension capabilities available on the device.
  val extensionsManager = ExtensionsManager.getInstanceAsync(
    application, cameraProvider).await()

  // Select the camera.
  val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

  // Query if extension is available. Not all devices will support 
  // extensions or might only support a subset of extensions.
  if (extensionsManager.isExtensionAvailable(cameraSelector, ExtensionMode.NIGHT)) {
    // Unbind all use cases before enabling different extension modes.
    try {
      cameraProvider.unbindAll()

      // Retrieve a night extension enabled camera selector
      val nightCameraSelector = extensionsManager.getExtensionEnabledCameraSelector(
        cameraSelector,
        ExtensionMode.NIGHT
      )

      // Bind image capture and preview use cases with the extension enabled camera
      // selector.
      val imageCapture = ImageCapture.Builder().build()
      val preview = Preview.Builder().build()
        
      // Connect the preview to receive the surface the camera outputs the frames
      // to. This will allow displaying the camera frames in either a TextureView
      // or SurfaceView. The SurfaceProvider can be obtained from the PreviewView.
      preview.setSurfaceProvider(surfaceProvider)

      // Returns an instance of the camera bound to the lifecycle
      // Use this camera object to control various operations with the camera
      // Example: flash, zoom, focus metering etc.
      val camera = cameraProvider.bindToLifecycle(
        lifecycleOwner,
        nightCameraSelector,
        imageCapture,
        preview
      )
    } catch (e: Exception) {
      Log.e(TAG, "Use case binding failed", e)
    }
  } else {
    // In the case where the extension isn't available, you should set up
    // CameraX normally with non-extension-enabled CameraSelector.
  }
}

To do this in Camera2, see the Create a CameraExtensionSession with the Camera2 Extensions API guide.

Implementing the Progress Bar and PostView Image

For an even more elevated user experience, you can provide feedback while the Night Mode capture is processing. In Android 14, we added callbacks for the progress and for post view, which is a temporary image capture before the Night Mode processing is complete. The below code shows how to use these callbacks in the takePicture() method. The actual implementation to update the UI is very app-dependent, so we’ll leave the actual UI updating code to you.

// When setting up the ImageCapture.Builder, set postviewEnabled and 
// posviewResolutionSelector in order to get a PostView bitmap in the
// onPostviewBitmapAvailable callback when takePicture() is called.
val cameraInfo = cameraProvider.getCameraInfo(cameraSelector)
val isPostviewSupported =
  ImageCapture.getImageCaptureCapabilities(cameraInfo).isPostviewSupported

val postviewResolutionSelector = ResolutionSelector.Builder()
  .setAspectRatioStrategy(AspectRatioStrategy(
    AspectRatioStrategy.RATIO_16_9_FALLBACK_AUTO_STRATEGY, 
    AspectRatioStrategy.FALLBACK_RULE_AUTO))
  .setResolutionStrategy(ResolutionStrategy(
    previewSize, 
    ResolutionStrategy.FALLBACK_RULE_CLOSEST_LOWER_THEN_HIGHER
  ))
  .build()

imageCapture = ImageCapture.Builder()
  .setTargetAspectRatio(AspectRatio.RATIO_16_9)
  .setPostviewEnabled(isPostviewSupported)
  .setPostviewResolutionSelector(postviewResolutionSelector)
  .build()

// When the Night Mode photo is being taken, define these additional callbacks
// to implement PostView and a progress indicator in your app.
imageCapture.takePicture(
  outputFileOptions,
  Dispatchers.Default.asExecutor(),
  object : ImageCapture.OnImageSavedCallback {
    override fun onPostviewBitmapAvailable(bitmap: Bitmap) {
      // Add the Bitmap to your UI as a placeholder while the final result is processed
    }

    override fun onCaptureProcessProgressed(progress: Int) {
      // Use the progress value to update your UI; values go from 0 to 100.
    }
  }
)

To accomplish this in Camera2, see the CameraFragment.kt file in the Camera2Extensions sample app.

Implementing the Moon Icon Indicator

Another user-focused design touch is showing the moon icon to let the user know that a Night Mode capture will happen. It’s also a good idea to let the user tap the moon icon to disable Night Mode capture. There’s an upcoming API in Android 16 next year to let you know when the device is in a low-light environment.

Here are the possible values for the Night Mode Indicator API:

UNKNOWN

• The camera is unable to reliably detect the lighting conditions of the current scene to determine if a photo will benefit from a Night Mode Camera Extension capture.

OFF

• The camera has detected lighting conditions that are sufficiently bright. Night Mode Camera Extension is available but may not be able to optimize the camera settings to take a higher quality photo.

ON

• The camera has detected low-light conditions. It is recommended to use Night Mode Camera Extension to optimize the camera settings to take a high-quality photo in the dark.

Next Steps

Read more about Android’s camera APIs in the Camera2 guides and the CameraX guides. Once you’ve got the basics down, check out the Android Camera and Media Dev Center to take your camera app development to the next level. For more details on upcoming Android features, like the Night Mode Indicator API, get started with the Android 16 Preview program.