Cameras and Digital Image Analysis

Top

00:00:11.00 Hi I'm Nico Stuurman. I'm a staff
00:00:15.00 scientist at UCSF and I prepared this
00:00:18.00 lecture together with Curt Thorn, who's
00:00:20.00 the director of the Nikon Imaging Center
00:00:22.00 at UCSF, and I want to start out with
00:00:25.00 presenting to you this nice complicated
00:00:29.00 new microscope system that we have in
00:00:32.00 the lab.
00:00:35.00 So here we have a microscope
00:00:37.00 system that we use in the lab every day.
00:00:41.00 Actually, people fight for time on this
00:00:44.00 system. You recognize here probably the
00:00:47.00 microscope body itself, the eyepieces,
00:00:50.00 there's the objective lens down there,
00:00:54.00 there is a motorized XY stage sitting
00:00:58.00 here that I can control with the
00:01:00.00 joystick or from the computer. Then here
00:01:05.00 on the sides we have a spinning disk
00:01:08.00 confocal unit and elsewhere in the
00:01:10.00 course it will be explained how that
00:01:13.00 spinning disk works. But then at
00:01:17.00 this, here we have a very fundamental
00:01:20.00 part of this microscope system and that
00:01:23.00 is a digital camera. So, this digital
00:01:26.00 camera is the thing that actually
00:01:29.00 registers the image. It collects the
00:01:31.00 image and then sends that image through
00:01:35.00 a cable to something else you will
00:01:39.00 recognize that is a computer. That
00:01:42.00 computer can then hold the digital image
00:01:44.00 in its memory or can write it to disk or
00:01:47.00 can display it on the computer screen.
00:01:51.00 And here in the computer we can do all
00:01:54.00 kinds of manipulations so we can
00:01:58.00 visualize the data, which is something
00:02:02.00 you always want to do, but we can also do
00:02:04.00 analysis, digital image analysis on these
00:02:08.00 images and extract quantitative
00:02:11.00 information about the things that we see
00:02:14.00 in a microscope. And so, in this lecture
00:02:16.00 I'll first be explaining to you the
00:02:19.00 principles of how a digital camera
00:02:22.00 works, and then also give you a very
00:02:24.00 basic introduction to digital images and
00:02:28.00 digital images analysis. So, we're going
00:02:33.00 to talk about cameras, and to just give
00:02:37.00 you another idea this is another camera.
00:02:40.00 It's a box, it has a bunch of electronics
00:02:43.00 in it, often it has something like
00:02:45.00 cooling, but the part that is really
00:02:48.00 important is shown here, and blown up
00:02:51.00 here in this figure. It's the chip, the
00:02:55.00 chip with photosensitive elements. So, it
00:02:59.00 contains rows and columns, rows and
00:03:02.00 columns of photosensitive elements that
00:03:06.00 we often call pixels, pixel elements, and
00:03:09.00 so, each of these pixel elements is able
00:03:13.00 to detect a flow of photons. Now to give
00:03:18.00 you a bit of a schematic overview in
00:03:20.00 blown up here, photons are coming in, and
00:03:25.00 by this semiconductor material, through
00:03:28.00 the photoelectric effect, are converted
00:03:30.00 into electrons. Those electrons are
00:03:33.00 stored for the duration of the exposure
00:03:36.00 in a potential well and then at a
00:03:39.00 certain point in time the number of
00:03:42.00 electrons is being read out and
00:03:44.00 transferred as a digital number to the
00:03:47.00 computer. Bunch of parameters with these
00:03:51.00 pixels that are of interest to us, so for
00:03:54.00 instance the most basic one is like how
00:03:57.00 large is a pixel, its width and height.
00:04:00.00 And we have an example here of the Sony
00:04:04.00 ICX285 chip, which is an CCD chip that
00:04:09.00 has been used now for years in
00:04:12.00 microscopy, the very popular chip design,
00:04:15.00 and that has pixels of about six and a
00:04:18.00 half microns square. The other number
00:04:21.00 that's important is how many we have. So
00:04:24.00 how many rows, how many columns? If you
00:04:27.00 actually multiply it up by this number
00:04:29.00 you get the physical size of the chip.
00:04:31.00 And so, for the ICX285 chip that has
00:04:35.00 about 1,400 columns and about a thousand
00:04:42.00 or so rows for a total of 1.4 megapixels
00:04:46.00 you see this is much less than most of
00:04:49.00 our cell phone or digit cameras or
00:04:52.00 digital SLRs have. So how many pixels do
00:04:58.00 we actually need in a microscope? And we
00:05:02.00 know that if you have fewer and fewer
00:05:05.00 pixels you get a block here, a block
00:05:10.00 here, a more pixelated image. And so, you
00:05:13.00 know that that is not good, so too few of
00:05:15.00 them is clearly not right. But if we have
00:05:18.00 more pixels, is there a point that it
00:05:22.00 doesn't make any sense anymore? And
00:05:25.00 clearly there is such a point and we now
00:05:29.00 think about that is what you try to do
00:05:32.00 is to match the pixel size that you have
00:05:35.00 to the resolution of your microscope
00:05:38.00 system. And so how do you match the two?
00:05:41.00 Well people in the signal processing
00:05:44.00 world have thought about this problem
00:05:46.00 for many many years and so a long time
00:05:49.00 ago they've come up with the so called
00:05:52.00 Nyquist Shannon sampling theorem and
00:05:55.00 that states that for the smallest limit,
00:06:02.00 the smallest distance that you can
00:06:04.00 resolve you will need two, at least two
00:06:08.00 digital samples. And so, in practice in a
00:06:12.00 microscope system that means that we
00:06:14.00 need two to three pixels to cover the
00:06:18.00 resolution limit of the microscope. So
00:06:20.00 what is the resolution limit of the
00:06:23.00 microscope? And so, you have seen in other
00:06:26.00 lectures that there are various criteria
00:06:28.00 for determining that and as an example
00:06:31.00 here I'm using the Rayleigh criterion. If
00:06:34.00 we now have a 100x lens that magnifies a
00:06:38.00 hundredfold with a numerical aperture of
00:06:41.00 1.4 NA, and we assume green lights
00:06:45.00 wavelength of 500 nanometers, then the
00:06:48.00 Rayleigh criterion
00:06:49.00 states that the resolution limit, the
00:06:52.00 shortest distance within which we can
00:06:55.00 still discriminate two-point objects, is
00:06:59.00 given by point 61 times the wavelength
00:07:03.00 of the light divided by the numerical
00:07:05.00 aperture. And that in this case leads to
00:07:08.00 a number of about 220 nanometers.
00:07:12.00 Now since we have a 100x lens that
00:07:15.00 means so we need to multiply it at
00:07:18.00 220 nanometers by a
00:07:20.00 100 in the camera plane that then
00:07:22.00 corresponds to 22 microns. Now according
00:07:26.00 to the Nyquist theorem we need at least
00:07:29.00 two pixels for every 22 microns and that
00:07:33.00 means that the pixels need to be 11
00:07:36.00 microns or smaller. And so, for instance
00:07:39.00 our ICX285 camera based camera has
00:07:45.00 six-and-a-half micron pixels so that
00:07:48.00 clearly would fulfill here the Nyquist
00:07:51.00 sampling rate. But when you, whenever you
00:07:54.00 buy a camera and want to match that to
00:07:57.00 your existing microscope these are the
00:08:00.00 kind of calculations you would want to
00:08:01.00 make. Now in microscopy, especially in
00:08:06.00 fluorescence microscopy, we barely ever
00:08:09.00 use color cameras and why is that? Well
00:08:14.00 that follows from the design of color
00:08:17.00 cameras. So, color cameras actually are
00:08:20.00 have the same kind of chip, a monochrome
00:08:24.00 chip, however in front of each pixel
00:08:28.00 element a small colored mask has been
00:08:32.00 put. So, for instance in this Bayer mask,
00:08:37.00 and that is what most RGB cameras use,
00:08:40.00 every four pixels will have two that are
00:08:43.00 have a green filter, one blue one, and one
00:08:47.00 red one, and that now means that when you
00:08:50.00 have a green photon hitting this Bayer
00:08:53.00 mask that only the ones hitting half of
00:08:58.00 these pixels will actually make it and
00:09:00.00 be detected. And the ones that hit
00:09:03.00 the other half will not be detected and
00:09:05.00 you lose them. So, a lot of light is being
00:09:08.00 lost and in fluorescence microscopy we
00:09:12.00 really like to put our filters inside a
00:09:14.00 microscope and we can design much better
00:09:16.00 filters than the ones over here. Of
00:09:20.00 course in things like observation of
00:09:24.00 histology sections, where you care much
00:09:26.00 less about light loss, color cameras are
00:09:29.00 totally fine but for fluorescence
00:09:31.00 microscopy you want to use a monochrome
00:09:34.00 camera. Another parameter in cameras that
00:09:37.00 is very important is their sensitivity
00:09:41.00 and the most important parameter there
00:09:44.00 is the quantum efficiency. And that means
00:09:48.00 of the photons that actually hit the
00:09:51.00 cameras, what fraction is converted into
00:09:55.00 an electron that I can then later read
00:09:57.00 out and actually see in my computer? The
00:10:02.00 quantum efficiency depends on the chip
00:10:05.00 design. So, for instance these back-
00:10:07.00 thinned CCDs
00:10:09.00 have a very high, can have a very high
00:10:12.00 quantum efficiency, up to 95% quantum
00:10:16.00 efficiency is also wavelength dependent,
00:10:18.00 so, you want to make sure that it matches
00:10:21.00 the dyes or the things that you're
00:10:24.00 looking at. Nevertheless, you know many
00:10:27.00 cameras nowadays will have between 70
00:10:29.00 and 90-95 percent quantum efficiency.
00:10:35.00 So, there are other sources of noise in
00:10:39.00 cameras. However, when we start talking
00:10:41.00 about noise the very first thing to
00:10:43.00 realize is that there is noise present
00:10:47.00 in any light signal and that is due to
00:10:51.00 the fact that your camera measures
00:10:53.00 photons. Photons are particles, so
00:10:57.00 particles will, even though the average
00:10:59.00 of a stream is constant,
00:11:02.00 they will not arrive at the pixel
00:11:05.00 element in a very regular way. There will
00:11:08.00 be a statistical
00:11:11.00 noise there and that statistical noise
00:11:15.00 is what we call shot noise. Shot noise
00:11:19.00 has a Poisson distribution and that then
00:11:22.00 dictates that the amount of noise scales
00:11:26.00 with the square root of the number of
00:11:28.00 photons and that means that if we have a
00:11:31.00 very bright signal, square root is
00:11:34.00 relatively small with respect to the
00:11:36.00 signal, whereas when we have a very dim
00:11:38.00 signal the square root is pretty high
00:11:41.00 with respect to the signal, i.e.: low amount
00:11:44.00 of light, just a shot noise will already
00:11:47.00 have a large effect, whereas when we have
00:11:51.00 a bright signal the shot noise is pretty
00:11:53.00 low. But you have to remember we can
00:11:56.00 never do better. Your camera can never
00:11:59.00 improve on the shot noise. The shot noise
00:12:01.00 is what comes with your actual signal.
00:12:03.00 And so, on top of that your camera will
00:12:07.00 add noise, and as a general rule of thumb
00:12:12.00 the faster the camera runs, the more
00:12:17.00 noise it will add. If you run a camera
00:12:20.00 very slowly, give it a lot of time to
00:12:22.00 read out its image, it can have a very
00:12:25.00 low noise. So luckily camera
00:12:29.00 manufacturers keep on working on these
00:12:31.00 kind of technical aspects. Then our
00:12:33.00 various schemes to deal with these
00:12:36.00 camera induce noise factors. So, for
00:12:40.00 instance their cameras are using
00:12:41.00 so-called electron multiplication gain,
00:12:44.00 their scientific see most cameras that
00:12:48.00 have very low readout noise. So that
00:12:52.00 means that there's a bright future and
00:12:54.00 good decent cameras will with low noise
00:12:57.00 will become more and more affordable and
00:13:00.00 easier to integrating to your microscope
00:13:03.00 system. So, enough about cameras and how
00:13:08.00 they work. Cameras generate an image, a
00:13:11.00 digital image in the computer. So, what is
00:13:15.00 a digital image? And I'm now giving you
00:13:18.00 here an example of an image of
00:13:20.00 diffraction-limited spots. So, these could
00:13:23.00 be
00:13:24.00 single molecules, it could actually as
00:13:26.00 well be an image of stars in the sky,
00:13:29.00 but in this case, they are single
00:13:31.00 molecules. I take this little area here
00:13:34.00 and we now zoom this up in the
00:13:36.00 computer and what you see is this little
00:13:40.00 pixelated area. So, we see these pixel
00:13:44.00 elements that actually corresponds to
00:13:47.00 the pixels that we had in the camera
00:13:49.00 originally. Now so this is a
00:13:52.00 representation of the image but what
00:13:55.00 does the computer now have in its memory?
00:13:58.00 And so, when we look at that we will see
00:14:01.00 that in its memory the computer has rows
00:14:05.00 and columns of numbers and these numbers
00:14:08.00 are then being matched to grayscale
00:14:12.00 values on the display. The actual image
00:14:16.00 though is rows and columns of numbers.
00:14:19.00 These numbers were measured by the
00:14:23.00 pixels in the camera and in the end
00:14:25.00 these numbers correlate with
00:14:28.00 the number of photons that hit that
00:14:31.00 photosensitive element and were caught
00:14:33.00 and converted in electrons, in photo
00:14:36.00 electrons. So, actually these numbers do
00:14:42.00 not map one-to-one to the number of
00:14:45.00 photons. However, a decent scientific
00:14:48.00 grade camera will have a datasheet that
00:14:51.00 gives you the so-called photon
00:14:53.00 conversion factor and with that photon
00:14:55.00 conversion factor you can map this
00:14:59.00 digital number in the actual number of
00:15:02.00 photons that were detected by that pixel,
00:15:05.00 which in the end is really what you
00:15:07.00 would like to know and how you can make
00:15:09.00 a fair comparison between different
00:15:12.00 images. Now, the important point again is
00:15:16.00 that your image is a matrix, rows and
00:15:20.00 columns of numbers that tell you how at
00:15:23.00 each of this pixel element how much
00:15:26.00 light was detected there during your
00:15:28.00 exposure. And what you want to be
00:15:31.00 absolutely sure of is that your software
00:15:33.00 is not changing those numbers because
00:15:36.00 that would be
00:15:37.00 changing your actual measurements and at
00:15:39.00 every point in time you want to be sure
00:15:41.00 what is going on with these numbers and
00:15:44.00 remember that the image is just a way of
00:15:47.00 displaying the numbers, your measurements.
00:15:52.00 Now, images can so we now know they
00:15:57.00 consist of numbers, they can have
00:15:59.00 different ranges of numbers. So, the
00:16:02.00 images we've been looking at so far are
00:16:05.00 so-called 8-bit images, they're contained
00:16:09.00 in 1-byte, and an 8-bit image 8 bits 8 zeros
00:16:16.00 have, can have 256 grayscale values.
00:16:20.00 That means we have 256 numbers and those
00:16:23.00 we map then to intensities on the screen.
00:16:27.00 However, you can also have an image that
00:16:29.00 has only 1-bit dynamic range and that means
00:16:33.00 then we have two grayscale values and
00:16:35.00 that is zero and black and this is what
00:16:38.00 we call a binary image. You can have an
00:16:41.00 image of these 2 bits, four grayscale
00:16:46.00 values, 4 bits 16 grayscale
00:16:49.00 values. With 16 grayscale values you can
00:16:52.00 see that it already starts to look a lot
00:16:54.00 like these 256 here. But there's no
00:16:57.00 reason to stop at 1 byte and just 256
00:17:01.00 grayscale values, many of the modern
00:17:04.00 cameras have a very large dynamic range
00:17:07.00 much larger than 256 so we need to
00:17:12.00 encode that in more bits and so we need
00:17:15.00 in at least 2 bytes and for instance
00:17:18.00 many cameras have a dynamic range of 12
00:17:21.00 bits 4096 grayscale values or 16 bits
00:17:26.00 where you have almost something on the
00:17:29.00 order of 65,000 different grayscale
00:17:32.00 values. Now that dynamic range also in
00:17:38.00 the end determines the image quality and
00:17:41.00 determines the information content of
00:17:43.00 the image. And just to give you an idea
00:17:45.00 of that, I encoded an image here in
00:17:50.00 2-bits, 4-bits, 6-bits and 8-bits. You see
00:17:53.00 that the 8-bit image looks like a real
00:17:57.00 image. We see all kinds of different
00:17:59.00 things here, we see detail in we see some
00:18:04.00 detail here in all those areas and that
00:18:08.00 detail completely disappears in this
00:18:10.00 image that consists of only four
00:18:13.00 grayscale values so the number of
00:18:16.00 grayscale values also determines the
00:18:20.00 information content of the image.
00:18:23.00 On the other end it's kind of interesting
00:18:25.00 that this 4-bit image already looks a lot
00:18:28.00 like the real image. Remember we have
00:18:31.00 only four grayscale values here, works
00:18:33.00 already pretty well and when you look
00:18:36.00 here four bits starts to look better but
00:18:39.00 this looks still quite pixelated but
00:18:42.00 here with the 6 bits, so that's just 64
00:18:47.00 grayscale values, looks here on my screen
00:18:50.00 almost the same as the 8-bit image.
00:18:54.00 So, the human eye and displays like this
00:18:57.00 actually cannot discriminate more than
00:19:00.00 roughly hundred different grayscale
00:19:03.00 values and that's why these 8-bit images
00:19:05.00 are often good enough to present
00:19:10.00 themselves very well.
00:19:13.00 So, as I already said
00:19:15.00 a monitor, the displays you use
00:19:19.00 on your phone, your computer, in general
00:19:22.00 has only can only display 256 different
00:19:27.00 grayscale values and that is fine
00:19:30.00 because your eyes
00:19:31.13 your accommodated eye
00:19:32.00 can actually not even discriminate each
00:19:35.00 of those from each other, but that also
00:19:39.00 implies something funny because if we
00:19:41.00 now have a 1-bit image it has a value
00:19:45.00 zero and a value of one. If we just put
00:19:47.00 that directly on that 8-bit display you
00:19:51.00 will see that both values are black.
00:19:53.00 So, the value zero is black and the value
00:19:56.00 one is black because the 8-bit display,
00:19:59.00 and we see that shown here,
00:20:01.00 is that all these low values will have
00:20:04.00 a very dark way of displaying.
00:20:08.14 and so conversely
00:20:10.06 if we have these images that
00:20:11.00 have like many many grayscale values,
00:20:14.00 so the 12-bit image for instance with 4096
00:20:19.00 grayscale values, everything higher than
00:20:22.00 256 if you display that on your monitor
00:20:25.00 will be white, you can't see it.
00:20:29.17 And so that means that we need a way
00:20:32.03 of mapping these pixel values that we have in
00:20:35.19 our image onto the monitor that can display it.
00:20:40.24 So, we don't want to change the
00:20:42.00 underlying values, we just want to change
00:20:45.00 the mapping from those values onto the
00:20:48.00 monitor.
00:20:49.23 Now so in practice the way you
00:20:52.00 do that is often by manipulating a
00:20:55.00 function that looks something like this.
00:20:57.00 So, we have a function that maps the
00:21:00.00 original intensity,
00:21:02.19 so here this is a 12-bit display image
00:21:06.17 going from 0 to 4095
00:21:09.23 and we map that to the display on the
00:21:12.00 monitor.
00:21:14.09 Now the simplest way of doing
00:21:15.00 that is to use a linear mapping.
00:21:20.00 So, we set 0 in the original image
00:21:22.22 to 0 on the display
00:21:24.21 and we set the highest value 4095
00:21:27.16 in the image to 255,
00:21:30.20 the highest value on the display.
00:21:33.08 So, if you do that
00:21:34.23 you have to be aware that you're not
00:21:36.00 seeing all the detail that you have in
00:21:38.00 the image. For instance, all the values
00:21:41.00 between 4080 and 4095 will all be
00:21:45.00 shown on the monitor as pure white as
00:21:49.00 255.
00:21:52.08 So often you want to manipulate this
00:21:54.00 function so that it brings out all the
00:21:57.00 detail that you have in the image.
00:21:59.27 And so, manipulating this function,
00:22:02.18 You often do by moving the maximum
00:22:06.24 or by moving the minimum.
00:22:08.24 And when you move each of these
00:22:11.06 by themselves what you change is the
00:22:13.27 slope of this line.
00:22:15.03 And when you change the slope of the line,
00:22:17.15 that will change the contrast that you
00:22:19.00 see in the image.
00:22:20.28 For instance, if you now
00:22:21.00 start to make lower pixel values white
00:22:25.00 then the brighter areas of your image
00:22:28.00 will get wider and wider whereas the
00:22:32.00 dimmer images of the area of the image
00:22:35.00 will stay the same so that will increase
00:22:38.00 the contrast.
00:22:40.07 So, when you move the minimum
00:22:41.24 and maximum simultaneously what
00:22:44.00 you will be doing is changing the
00:22:46.00 brightness of the image.
00:22:48.24 So, another tool
00:22:49.00 that is very useful when you are
00:22:53.00 manipulating image display is the
00:22:56.00 histogram. So, a histogram will display
00:22:59.00 for every grayscale image, for every
00:23:02.00 grayscale value in your image, how many
00:23:06.00 pixels there are that have that specific
00:23:09.00 gray scale,
00:23:10.22 that that have that specific number.
00:23:13.09 So, we have here on the y-axis the
00:23:15.00 number of pixels,
00:23:17.10 on the x-axis is the pixel value.
00:23:21.02 You see here that at this
00:23:22.00 specific grayscale value we have a lot
00:23:25.00 of them, so that's probably the
00:23:27.00 background here in our image.
00:23:30.01 And so often you use this histogram
00:23:32.22 to match the maximum and minimum.
00:23:36.20 You set the maximum to the
00:23:38.02 maximum pixel value in your image,
00:23:39.29 the minimum to the minimum
00:23:41.04 pixel value in your image
00:23:43.02 you already have a very good
00:23:44.06 starting point to look at
00:23:47.26 what is going on in your image.
00:23:52.19 just to give a few examples we have here
00:23:54.00 another drosophila spindle image and we
00:23:57.00 now scale this to the minimum and
00:23:59.00 maximum as I described.
00:24:01.22 You see the kinetochore fibers
00:24:05.08 and you see some detail here
00:24:06.25 elsewhere in the cell.
00:24:08.15 You can now make that come out a bit more
00:24:12.00 by moving that maximum so that makes
00:24:14.16 the image brighter,
00:24:16.20 it increases the contrast.
00:24:20.00 If we now move also the minimum
00:24:21.00 here then you see that the darker areas
00:24:25.00 of this image are totally being clipped
00:24:28.00 off and it disappears.
00:24:31.05 So, we have here an even
00:24:32.00 higher contrast and we start to lose
00:24:34.00 some of the information here but we
00:24:36.00 possibly can make out a bit more of
00:24:39.00 what's going on with these brighter
00:24:41.00 kinetochore fibers.
00:24:43.16 So, each of these ways of displaying
00:24:46.20 the data is in a way equally valid.
00:24:50.20 When you compare images
00:24:51.00 with each other it's very important that
00:24:54.00 you use the same brightness contrast
00:24:56.00 settings otherwise you don't know what
00:24:59.00 you're comparing with each other.
00:25:01.00 And in general, you really should be sure that
00:25:04.00 if you want to use your images to make a
00:25:06.00 point to others that you describe how
00:25:09.00 you are displaying the data.
00:25:13.01 Quick word about color images.
00:25:15.14 So, color images in the computer
00:25:18.03 are basically treated as individual grayscale
00:25:23.03 images, where we now have for an RGB
00:25:27.07 data set, we have a red image,
00:25:28.28 a green image and a blue image.
00:25:30.00 Each of the in the computer memory sees
00:25:33.00 it as rows and columns of numbers that
00:25:36.00 then where we just arbitrarily say ok
00:25:39.00 this one is going to be displayed red,
00:25:41.00 this one green, and this one blue and
00:25:43.00 when we combine that all together we get
00:25:46.00 back our RGB image. And just like
00:25:49.00 monochrome images, RGB images can be
00:25:52.00 stored within 1 or 2 bits per channel.
00:25:56.28 Most consumer grade channels
00:25:58.16 will be 1 bit per channel but when
00:26:01.00 you go to SLRs with raw images or
00:26:04.00 professional-grade
00:26:05.00 RGB cameras they can have a much larger
00:26:08.00 dynamic range and use two bits per
00:26:11.00 channel.
00:26:13.24 File formats
00:26:14.26 So, once you have your data in the computer
00:26:17.18 you want to store them one way or another
00:26:20.07 on disk such that you can also exchange
00:26:22.26 them with other people.
00:26:24.11 What is important to remember here
00:26:27.00 is that many of the file formats
00:26:29.08 will try to compress the data for you.
00:26:32.16 Compression is nice because it
00:26:34.00 reduces the amount of data that will be
00:26:39.00 taken up on the hard drive.
00:26:41.00 However, some forms of compression will
00:26:46.00 no longer allow you to restore the
00:26:49.00 original data and that is bad
00:26:52.00 because you won't later be able to know
00:26:55.00 exactly what the camera has measured.
00:26:58.13 And so therefore we call
00:27:02.07 have terms for this:
00:27:03.04 One is lossy compression.
00:27:05.21 So lossy compression we lose data,
00:27:08.04 we cannot get it back again.
00:27:09.28 Lossless compression is a
00:27:11.00 compression where you do not lose data
00:27:13.00 so you have a way of restoring the exact
00:27:17.00 original image content.
00:27:19.25 So lossless compression is okay,
00:27:22.17 lossy compression is not good
00:27:24.27 if you want to continue your analysis.
00:27:28.16 File format that is widely used
00:27:30.00 is the so-called TIFF format.
00:27:34.09 TIFF is a container format, there are many
00:27:36.00 different ways of storing your data
00:27:38.00 within a TIFF. You can have 8 or
00:27:40.00 16-bit data, you can have, it can
00:27:44.00 incorporate lossless compression,
00:27:47.00 it supports grayscale as well as RGB images
00:27:50.00 and in general TIFFs are really a nice
00:27:52.00 way to store your data and you can
00:27:55.00 exchange it easily with others.
00:27:58.19 There are a bunch of, there's a whole lot more
00:28:00.00 actually than these more proprietary
00:28:03.00 data formats.
00:28:06.16 Some of them more open than others.
00:28:08.19 Ones that you will also know are
00:28:10.00 things like JPEGs, GIFs, BMPs.
00:28:14.00 And in general, this class of file format is
00:28:18.00 not great to store your scientific grade
00:28:21.00 data. For instance, JPEGs use a lossy
00:28:25.00 compression scheme, bad because you
00:28:28.00 cannot restore your original data and
00:28:31.00 others like GIFs and BMPs can
00:28:34.00 only store 8-bit data so that's already
00:28:36.00 out when you have 12 or 16-bit images.
00:28:41.00 Now one project that needs, that deserves
00:28:45.00 mention here is the Bioformats project.
00:28:50.00 And so, the people behind this have
00:28:52.00 decided to make a
00:28:54.00 tool that is able to read all these
00:28:56.00 different file formats but also the
00:28:59.00 hundreds more that are out there and that
00:29:02.00 a different microscope company and
00:29:03.00 camera companies have come up with and
00:29:05.00 so with this Bioformats that's used in
00:29:08.00 many different software packages
00:29:09.00 including image J, you can read more or
00:29:13.00 less anything that you encounter.
00:29:17.00 So, super useful software.
00:29:20.04 Now when you have your digital image
00:29:22.20 you will want to measure something
00:29:25.18 and often the following way is taken to do so
00:29:31.21 It is done by segmentation
00:29:34.00 and in segmentation what you do is to create
00:29:38.00 a mask of things that are in
00:29:40.17 and conversely things that are out.
00:29:43.24 So that mask is a binary image
00:29:48.17 So, it just contains one bit per
00:29:51.00 pixel and when it's one that means it's
00:29:54.00 in and when it's zero that means it's
00:29:57.00 out so it's background.
00:29:59.19 And the easiest way of segmenting is to look at
00:30:03.01 your grayscale values and move a slider and
00:30:06.00 say everything below this specific
00:30:09.00 grayscale value is out and everything
00:30:12.00 higher than this grayscale value is in.
00:30:14.00 And that's how you can do a segmentation
00:30:17.00 like this, where you see that these spots
00:30:19.00 correspond pretty nicely to what you see
00:30:22.00 in your dot blot.
00:30:24.04 There's also some junk here,
00:30:27.18 little things and there's some
00:30:29.00 binary manipulation techniques: erosion,
00:30:33.00 dilation, where you wrote your pixels in
00:30:36.00 and then dilate them back out again so
00:30:39.00 they restore the original. But then
00:30:41.00 things that are too small simply erode
00:30:44.00 out and disappear. You can see that you
00:30:47.00 can nicely clean it up and now the
00:30:49.00 spots correspond very well to the spot
00:30:51.00 in the dot blot.
00:30:54.25 So, measuring objects then
00:30:57.00 often consists of starting out with the
00:31:00.00 image, do a segmentation so that you get
00:31:03.00 masks that correspond to the original objects.
00:31:06.00 You know have each of these masks you put
00:31:10.00 over the original image and you ask what
00:31:13.00 were the grayscale pixel values there.
00:31:16.00 You can get out the total number of
00:31:18.00 pixels, the mean, the integrated density.
00:31:22.00 You get then a list with the intensity
00:31:26.00 under each of these objects and that
00:31:29.00 already can give you a very clear idea
00:31:31.00 of what is going on in this image.
00:31:34.00 Now this is a very simplified way to show
00:31:39.00 you how measurements work.
00:31:42.25 Just to give you an idea,
00:31:45.07 there's more and more image
00:31:46.00 analysis being used in biology.
00:31:49.00 For instance, this is here a very recent
00:31:53.00 publication in Nature Methods,
00:31:55.00 where the authors developed a method to
00:31:58.00 automatically determine sites of mRNA,
00:32:04.00 mature mRNA as well as nascent RNA in
00:32:08.00 images obtained by fluorescence in situ
00:32:11.00 hybridization. So, it goes to show how an
00:32:14.00 automated image analysis procedure can
00:32:17.00 give you very quickly and easily
00:32:19.00 biological insight extracted out of the image.
00:32:23.17 And another example I want to show
00:32:25.00 you comes from this RNAi screen that we
00:32:30.00 did, that our group did a few years ago
00:32:32.00 in Drosophila S2 cells, where we were
00:32:36.00 looking for genes affecting mitotic
00:32:39.00 spindles, and where we used these kind of
00:32:41.00 image analysis techniques to
00:32:43.00 automatically detect a mitotic spindle
00:32:46.00 in the images that we acquired, and to
00:32:48.00 measure various aspects like for
00:32:51.00 instance the distance between these poles
00:32:53.00 to get an idea of the spindle length.
00:32:57.20 Now, I hope you got a basic
00:33:00.00 understanding of how digital imaging in
00:33:04.00 microscopy works.
00:33:06.07 And you know, go ahead
00:33:08.00 and play with these techniques yourself
00:33:11.00 and have a lot of fun with it.
00:33:14.00 Thank you.

This Talk

Speaker: Nico Stuurman

Audience:

Student
Educators of Adv. Undergrad / Grad
Researcher

Recorded: April 2013

Talk Overview

In this lecture about digital image analysis, Nico Stuurman describes how digital cameras for microscopes work, what a “pixel” is, Nyquist sampling, the dynamic range, noise, and color cameras. The features of an image are described, included bit-depth, intensity scaling and histograms, file formats and basic image processing.

Questions

According to Nyquist sampling, which camera pixel size will be a reasonable match for a microscope with 250 nm resolution and 100x magnification?
1. 25 μm
2. 2 μm
3. 8 μm
4. 250 μm
5. 50 μm
How many grey scale values are there for a 12 bit image?
Which of the following file formats is the worst if you want to store a 8-bit image without loosing information?
1. JPEG2000
2. JPEG
3. GIF
4. TIFF
Camera noise will decrease with faster readout rates. True or False.
Which of the following is false?
1. Photon shot noise scales linearly with the number of photons
2. Quantum efficiency in a ccd camera is the the probability in which an absorbed photon will be converted to an electron.
3. Color cameras are less light efficient than black-and-white cameras.
4. A 16 bit image must be scaled to 8 bit to be displayed on a monitor.
An intensity histogram displays how many pixels have a particular grey scale value in an image. True or False?

Answers

View Answers

Speaker Bio

Nico Stuurman

Nico Stuurman is a Research Specialist at the University of California, San Francisco, in the lab of Ron Vale. Nico combines his expertise in computer programming and microscopy to advance many projects including the Open Source software, Micro-Manager. Continue Reading