[Let's Talk Imaging] The Future is AI

dipankar paul

Hello everyone,

Welcome to Round 3 of our Let's Talk Imaging debates.

We've had a good conversation in our first and second debates. You've made your opinions known on whether or not more megapixels and camera sensors actually enhance smartphone photography, and whether you prefer a true-to-life output on your smartphone or social media-ready photos.

It's time now to discuss the role of Artificial Intelligence in smartphone camera systems.

Does AI hold the key to future of smartphone photography? What applications of AI do you foresee in the near term and the long term in the realm of smartphone photography?

The floor is open. Let's talk!

krishnasnapshots

A smartphone is made to do lot of things on fingertips & that too without hesitation, just click and move. So, in these terms, I do feel AI hold the key to future of smartphone photography. Like Single take feature in Samsung phones where, user get full advantage to shoot video, photo, slo-mo, time lapse, etc. in one single click & later decide what to keep without losing a single moment.

praveenp

Having spent more than a year on studying image processing using AI, particularly SVM and CNNs, during my Post Graduation dissertation work that included theory and programming both, it gave me a great opportunity to learn about the expanse of AI and image processing, which had always been a field of my interest.

The future of imaging surely lies with Artificial Intelligence or AI. The image/ scene recognition algorithms are like an asset now for mobile computing systems.

The earliest AI-like camera features were mainly limited to smile detection or gesture-based clicking, which were present in phones like those by Samsung (since I had used one, so I know. There may be many others too).

But, a couple of years back, in around 2018 when Huawei introduced AI in camera, with the introduction of dedicated hardware in the form of neural engines in the SOCs, AI made a dedicated breakthrough in smartphones. Though it was mainly restricted to cameras for assessing the scene and switching to a compatible mode, but it opened a lot of avenues to explore in this direction. This was further enhanced to make the modern phones more adaptive towards optimizing the smartphone's performance based on the usage, which has now become a core part of newer Android versions.

But when it comes to camera systems, these 2 years have taken a massive leap. The camera software that was having limited AI functionality then, has now become very much dependent on AI.

The features of scene detection have now upgraded to selecting the best shot from a plethora of images captured at once, or capturing the best night mode shot with maximum details, object detection, and even blurring the background.. We can see it working in Google Photos app which analyses every photo and categorizes it based on person, place, scene, occasion, etc.

Since the technology of Augmented Reality is now taking up pace, we will surely see better and out-of-the-box advancements for AI in smartphone cameras.

The modern systems where the world is connected though smartphones, imaging will play a crucial role in almost every field.

The prominent ones which I see in the near term and long term future are:

1. Online Shopping: instead of trying the clothes physically, AI-based models will enable trying the virtual clothes just via camera (like what lenskart does by allowing users to try frames).
2. Video Shooting: we can expect a more efficient EIS (electronic image stabilization) in near future, which might be at par, or even better than OIS (optical image stabilization).
3. Tele-medicine: though robotics is now taking the position of humans for complicated surgeries, some medical examinations do require human intervention for a perfect result as humanoids like Chitti are still a talk of distant future. We might see ultra-sound images, or MRIs and X-Rays getting more smartphone-friendly and allowing precise extraction of relevant data from patient reports just at the reach of the doctors' pockets.
4. Improved Security: phone security has now become of immense importance. With **** recognition becoming a common feature of security, we will be seeing much more improvements based on AI. The cameras of future smartphones might be able to detect minor **** features and expressions of the original owner and a doppelganger who might try to have an unauthorized access to someone's phone.
5. Lossless Imaging: algorithms like CNNs can very well re-construct images using deep learning mechanism. We will surely be seeing a better software optimization of images whose quality gets deteriorated to some reasons like digital zooming, shakes, etc.

Apart from these, since the languages like Python, R, Hadoop, etc, are becoming more and more programmer-friendly by addition of dedicated AI packages, it will give more flexibility to new-age developers to explore the ocean of features that can be added to smartphone imaging.

singhnsk

@praveenp What kind of recognition were you talking about that the system had to censor? 🙈

praveenp

https://community.phones.nokia.com/discussion/comment/176565#Comment_176565

https://community.phones.nokia.com/discussion/comment/176555#Comment_176555

@singhnsk The censored word is fac!al (FAC!AL)...

Don't know why it has been marked as offensive? 🙄

nokianchetan

Definitely AI hold the key to future of smartphone photography. I have seen so many people who are clicking better photographs better than before with the help of AI

Most of the people clicks pictures of Landscape , Flowers and Portraits & AI helps them to enhance the pictures according to scene

But i want to see the AI to recognise the subjects distance so that it can zoom in or out accordingly .

Parvesh Khatri

What I think is that we should get to choose how we want to capture. AI is good for people who just want a straight good looking image, ready to post on social media. For people who're in mobile photography, RAW image is best output. Photography is not less than painting. We can teach computers how to edit some aspects of a picture. Auto-edited pics can be good, can be bad or average looking. True photography can transfer feelings that the photographer was experiencing. These feelings were brought by editing by human. AI images can be good looking, but they will never ever give the same vibes as manually edited pics. That's because computers don't feel anything except temperature of the processor. Most of the times details in a photograph does not matter that much, but looking through the photographer's eyes matter. I saw many phones auto edit photos even when taken in pro mode, then what is the point of a pro mode?

When it comes to capturing some moments quickly, pro mode sucks. Humans can't adjust that many controls fast enough that the moment does not pass away. Here the important role of AI comes, AI should adjust the focus, ISO and shutter time. Then it should produce two outputs RAW and JPEG. A fast storage may become helpful in this.

Role of AI can be seen in best way from a video-maker prospective, who is shooting a horror scene with parts of humor involved. If AI don't have any control, he has to take smaller clips as he cannot adjust the controls to change in brightness fast enough when the brightness of scene is changed. If the AI controlled everything, then there will be no difference between horror and humor. AI should have control over basic controls only.

madbilly

Hi all,

Interesting subject! But how come we can only discuss this in India? Come on @dipankar paul, please post these interesting discussions where the rest of the Nokians might find them 😉

I have mixed feelings about "AI". Certainly, we can see good and bad examples of how it "interferes" with photography. My most recent experience is only with a Nokia 8, which does have some "AI" but generally it doesn't really have it.

What it does have is scene recognition, but this tends not to work very well. It is used to determine whether or not to use HDR or flash (and of course adjust the various exposure, ISO, shutter speed, etc settings), but it often chooses to use HDR when there is a moving subject and when it's using flash it's just to slow to capture the moment. In cases of lower light or moving objects I therefore turn these off, but I would prefer not to, I would prefer to leave them turned on.

So far the scene recognition for choosing other settings seems to work quite well. There are of course cases where it's not so good (white balance is an obvious one here) bu in those cases I just take another photo if I can.

You'll note that I said "AI" in inverted commas. That's because often it's just a catchphrase that is used without anyone knowing what they're really talking about. Some of AI is machine learning, sometimes with neural networks. I'm not sure exactly what is being achieved with this, maybe it's better scene recognition and adjustment, certainly it's recognition of objects in the frame and adjusting focus, white balance etc. However, some of this is where we see the worst of AI - where it starts to change what is actually being photographed and remove things or add things that were never there!

Edge detection and sharpening, or the opposite, is usually to blame for this. Remember the 9PV comparison shots of the sky which showed loads of clouds but the competitors phone (a Samsung I think) barely showed any clouds at all? I also saw a comparison on AAWP where the 9PV clearly showed clumps of snow on the ground but the others (including the Lumia 950) didn't show this. Now, I didn't take the photos, so I don't actually know which photos were telling the truth, so if AI is making adjustments like this what else is it doing?

Oversaturation of colours is another horrid case! How many times do we see a photo in a review which shows the sky as blue but actually on that day it was grey!

A case where AI removes things to the perceived benefit of the user is "portrait" mode, where all those blemishes and things are magically removed! 😆

So, in summary:

• Yes please use AI for scene recognition and to automatically adjust the settings to capture the moment
• No please don't use AI to add and remove things which materially changes what the photo and makes it different from reality.

Cheers 🙂

nitinkachare

I like AI features, such as scene detection, face detection, portrait, AI portrait lighting, night mode, gestures detection, smile detection, blink eye detection, etc. AI enhance the image quality and get the image as much as possible details and well-saturated colors according to the subject and different scenarios. Sometimes, AI shows the wrong scenarios while capturing images. AI works with neural processing engine so it's getting better over the time by the machine learning algorithm. AI which works in combining multiple images into one good image is nice. Different AI Bokeh Effects captured with the use of different sensors like tof and depth is a nice thing.

AI colour feature(black and white effect) and AI background blur is a very good thing in imaging as well as in video recording. In Lumia Camera, I have seen some of feature i.e. Rich capture,Live Images, Smart Group Shot, Action Shot, Blink Eye Gesture, a red-eye removal which was quite good at that time.