Help me understand why rolling shutter effect happens
Help me understand why rolling shutter effect happens
Hi,
I'm trying to wrap my head around the rolling shutter effect, specifically why it happens.
I'm having a hard time understanding how the readout speed affects the image. If I understood correclty, when in electronic shutter mode the pixels are exposed as indicated by the shutter speed (e.g. at 1/1000 each pixel is exposed for 1/1000 of a second).
If the readout takes 1/100 s to scan the entire sensor, what happens exactly when I take the picture? Do the pixels start firing sequentially as the shutter speed dictates (i.e. 1/1000 s each, sequentially)? If that is the case, do they wait for the readout to catch up or do they continue firing? If the latter, by the time the readout reaches the second pixel, the eleventh pixel is firing, so there are 10 pixel between the one firing and the one being read. Does it work like this?
If the pixels are exposed for 1/1000 s and then turned off and their value stored, wouldn't that mean that the image should not be affected? I mean, they saw the subject for 1/1000 s and the motion should be frozen, they are just waiting for the value to be read. Just like if you asked 10 people to open their eyes for 1 second (shutter speed), one after the other, and draw what they see. They saw if for one second each, so at most the difference in the position of what they saw should cover 10 seconds. Then they can take hours to draw what they saw (readout speed), but what they saw specifically wouldn't be afftected by how long it takes them to draw it. Am I wrong here maybe?
Also, in general, why is mechanical shutter not as affected (if affected at all) by the rolling shutter effect? Does the sensor capture light differently when in mechanical shutter mode?
I just don't get it. I feel like I'm close to understanding why, but I still don't.
I know I'm probably weird for focusing so much on something technical like this, but it just bugs me so much.
Any help is greatly appreciated, really.
I could try but honestly Destin at Smarter Everyday will do such a better job than me. I also included a slow mo guy vid on the same subject. It has been a bit since I watched either of these vids so I hope they answer your question.
9Thanks for the videos, I love both of those channels! But they don’t really explain the intricacies behind the rolling shutter, so they don’t answer my specific questions :(
1
Another good video from Captain Dissillusion:
9This is a great video! You made me discover such a good channel. However, it gets close but not enough :(
2Try holding your hands a few cm/in in front of your face. Fork your fingers to make a narrow slit you can see through. If you move your hands from top to bottom you will be scanning only part of the scene at a time. So if something is moving (car, child, etc.) it will be in a different place when your "slit" is at the bottom than when it was at the top.
3
Most camera sensors can only be read one line at a time. This means that the second line isn't captured until the first line is completed. This goes for all of the remaining lines on the sensor of which there are thousands. For example a 24 megapixel camera might have around 4000 lines of pixels to capture.
The image processor captures the state of the top line and moves on to the next. By the time it's captured the 2nd line, if the subject has moved, that line is no longer aligned with what was captured in the first line. Continue this 3998 more times and you have a frame of a video where the bottom of the frame is further ahead than the top of the frame.
To slow this down to a human understandable scale, while watching a video close one eye, then open it and close the other. By the time you've opened the second eye and closed the first things have moved. Just don't do it while driving.
5The basics: the photo sites on a camera sensor can be though of as buckets with a color filter on top of them, so each site captures only red, green, or blue light. When you take a picture the site is emptied, filled for the exposure duration, and then measured. The fuller the bucket the more intense the light was at that spot on the sensor. Buckets have a finite capacity, so if you fill them too much you'll reach "full" and blow the highlights.
Electronic shutters fall into two camps: progressive (also called rolling) and global.
On a global shutter sensor, all photo sites are emptied, filled, and read simultaneously.
On a progressive shutter, this happens row by row so you empty, fill, and measure the first row, then the next, then the next. Even if each photo site is only exposed for 1/10000 of a second, the process of reading all the photo sites can take a while so the "time the picture was taken" progressivly gets delayed as you go across the sensor. On Sony's A7R V this process takes about 1/10 a second. Even if your exposure is set to 1/10000 of a second, the process of taking the photo still takes 1/10 second. If anything was moving at a decent clip during the exposure it will look stretched, because the photo is essentially a series of row-by-row photographs that get stacked together over that 1/10 a second. Faster sequential sensors, like the Nikon Z9, can achieve 1/270 readout speeds which minimize, but don't eliminate, this effect.
It's worth saying that mechanical shutters are also progressive shutters and that a full frame mechanical shutter takes somewhere between 1/200 to 1/250 to traverse the sensor. This is why the Z9 doesn't have a mechanical shutter - there isn't a ton to be gained from having one. Mechanical shutters are significantly faster than the electronic shutters on most cameras, but it's still possible to get the rolling shutter effect with a mechanical shutter if you're planning quickly or taking a photograph of something moving fast enough.
The only way to eliminate the effect completely is with a global shutter.
2Thanks for getting into the details.
I'm still not 100% there...
I still am not sure how the exposure is started: is it row by row or is it the entire sensor and it's just the readout that is sequential? In the first case, then it would make sense to me because they would gather 1/1000 s of light and then, even though the readout is slow, they are physicall not receiving any more light because the shutter is closed and it's dark, so the image is not affected. If the exposure starts sequentially as well at the readout speed, I would not understand how it could keep up with the curtains, because the curtains would fly over the sensor while the pixels didn't have the chance yet to be "activated", so only the first few pixels would see something and the rest would be exposed when the curtain is already closed.
When in electronic shutter though, the pixels must not be activated all at once because then the first row would get less light than the last rows, and this also would mean that the exposure itself is the readout speed which is not the case. So they are activated sequentially, which is in contrast with how it works for mechanical shutter. In this case though, like in my post, how can the image "move" if the pixels are exposed for only 1/1000 s? Is it because the next pixel is exposed for 1/1000 s only after the first one is read, so there is a delay between one pixel being exposed and the next one? Like pixel #2 is not firing for 9/1000 s because it's waiting for the first one to be read by the 1/10 s readout.
What am I getting wrong here? I'm sure there is some misconception in my mind that is preventing me to see what is going on clearly like everyone else.
1I still am not sure how the exposure is started: is it row by row or is it the entire sensor and it's just the readout that is sequential
The exposure is always started row by row for a progressive (electronic or mechanical) shutter. It's also read out row by row.
Really truly think of the photo sites as buckets collecting liquid light. It seems like you understand that they all need to be exposed for the same duration to get an even exposure. That doesn't mean they all need to be read immediately - these buckets don't leak. In other words, it's OK for a mechanical shutter to sweep the sensor much faster than the sensor can be read. Those buckets will still have the correct quantity of light in them when they're (eventually) read.
Step 1: empty the buckets in the dark
Step 2: expose the buckets for the same time duration. Note that this ends with the sensor back in complete darkness
Step 3: measure how full the buckets are
No light hitting the sensor at the end of step 2 means the buckets won't keep filling while they're waiting to be read.
I would not understand how it could keep up with the curtains
For a mechanical shutter it doesn't have to.
So they are activated sequentially, which is in contrast with how it works for mechanical shutter.
For both electronic and mechanical sequential shutters, the rows are still exposed in sequence (the top most row gets its exposure started before the row below it). It is a sequential shutter after all. In the case of a mechanical shutter, this takes 1/250, which is pretty fast but it's not instant.
In this case though, like in my post, how can the image "move" if the pixels are exposed for only 1/1000 s? Is it because the next pixel is exposed for 1/1000 s only after the first one is read, so there is a delay between one pixel being exposed and the next one? Like pixel #2 is not firing for 9/1000 s because it's waiting for the first one to be read by the 1/10 s readout
The time delay between exposiyand reading rows is exactly what causes objects to look stretched. The only difference is that sweeping the shutter with the mechanical shutter happens 25 times faster than the electronic one.
Ask more questions, but I think you're close!
2
The photosites are continuously reading the light falling on them. When you take a picture, the system takes discreet readings of those values for the length of your shutter speed. However it can only handle the information from a portion of the sensor at a time. So it reads in sequential stripes. The longer the exposure time the more of a chance that the sensor or the subject has moved before the camera reads all the stripes.
2Thanks for the reply. May I ask you to go into more details?
1
You know how a strobe light can make things look like its sort of jumping through time? Sort of make it look like there's a way lower frame rate irl? It's like that but imagine instead of the light flashing on and off, it rolls from top to bottom.
1