Introduction to Digital Images

Top

00:00:11.07 I'm Kurt Thorn from the Nikon Imaging Center at
00:00:13.21 UCSF, and today I'm going to talk about digital image
00:00:17.00 analysis. And this is part one of a two part set of lectures.
00:00:20.28 And in this part I'm going to talk primarily about what
00:00:24.05 digital images are, how you display them on your computer,
00:00:27.15 and how you can save them so you can work with them later.
00:00:31.11 So what a digital image is, essentially, is a set of measurements
00:00:36.22 of light intensity. So we have a camera, we've talked about cameras
00:00:40.07 in other lectures, or a confocal microscope or some device
00:00:43.24 that records light intensity at a set of points in your field
00:00:46.25 of view in your microscope. So you get an image like this one
00:00:50.12 here, where you've got a bunch of bright objects in a field.
00:00:55.10 And if you zoom in one of these objects, you can see the individual
00:00:58.03 pixels that make it up here. Some of them are bright,
00:01:00.28 some of them are dark. And what each of those really represents
00:01:03.21 is a measurement of the light intensity that was recorded
00:01:06.16 at that point on the camera, or at that point in the image.
00:01:08.19 And so if you look at the numbers that make this up,
00:01:11.06 you can see here this array of numbers that corresponds to
00:01:15.24 pixel intensities in each one of these pixels. And you can see
00:01:19.13 that they range from zeroes here at the edges, where it's
00:01:22.12 totally black, to you know, 200-255 in the center, where it's
00:01:26.15 very bright. And so these images are just really an array of
00:01:32.20 numbers which represent the light intensity at each point
00:01:34.22 in the field of view. And computers store numbers in binary
00:01:40.13 format. And in binary, you can have a large number of different
00:01:46.14 representations, and depending on what resolution or what
00:01:49.16 dynamic range you want, you can start with a different number of bits.
00:01:53.03 So, a single bit here at the top can represent zero or one.
00:01:56.16 So this would be a binary image, it's either black or white.
00:01:59.28 If you do 2 bits, now you can have four colors like this bar here.
00:02:03.24 Now you've got black and white plus two shades of gray.
00:02:07.08 And you can keep adding bits to get more and more shades.
00:02:10.25 If you do 4 bits, you get 16 shades of gray. If you do 8 bits,
00:02:14.17 you get 256 shades of gray. And you know, you can do 12
00:02:19.04 or 16, which will get you 4000 or 65000 shades of gray.
00:02:23.03 Computers for historical reasons, tend to store numbers
00:02:28.17 as bytes, and a byte is 8 bits. So these final formats here on the
00:02:33.11 top, or these bit depths on the top, can be represented as a
00:02:36.12 single byte. So one byte will store one pixel's worth of
00:02:39.29 information. Whereas below this line here, when you go
00:02:44.04 above 256 and you have more than 8 bits, now you need 2
00:02:46.29 bytes. So you would store these generally as 2 bytes or
00:02:50.06 16 bits. And so the higher the intensity resolution
00:02:56.06 you want, the more dynamic range you want, the more
00:02:58.10 space it takes to store your data. But also the more fine
00:03:02.24 resolution you get of different gray scales of the image.
00:03:05.25 Here I'm showing an image at a number of different bit depths,
00:03:09.10 and that the top left corner here is the original image, which is
00:03:12.20 an 8 bit image. It's got 256 gray scales. And then these other panels,
00:03:19.00 I've reduced the number of gray levels and reduced the bit depth to
00:03:22.19 smaller numbers. So down here, you can see what happens
00:03:25.16 if we make this into a 6 bit image. And we have a fewer number of
00:03:30.08 gray levels in the image, but it's actually very hard to see the
00:03:32.24 difference between these two. And that's partly because your
00:03:35.06 computer monitor can only show 8 bits of gray scale
00:03:39.02 and partly because your eye is not sensitive to very many
00:03:43.04 shades of gray. Your eye at best can resolve 100 or maybe
00:03:46.29 150 gray levels. But then when we start reducing the bit depth
00:03:51.09 further, if we go to a 4 bit image here, you can now start
00:03:53.24 to see this sort of posterization, where you get, especially in the
00:03:57.12 smooth gradients, large discontinuities in the intensities
00:04:00.20 because we don't have enough gray levels to make a smooth
00:04:02.19 shading. And in 2 bits here, we see not only 4 grayscale,
00:04:05.25 but we get a very stylized image that doesn't bear a lot of
00:04:09.10 resemblance anymore to the original intensities here.
00:04:11.03 So the more bits you have, the more gray levels you can
00:04:14.17 record. And modern cameras can quite easily record
00:04:17.20 more gray levels than you can resolve with your eye.
00:04:20.02 And as I mentioned, your monitor, your computer monitors
00:04:24.07 are 8 bit displays. So they can only display 8 bits of gray
00:04:29.17 scale information, so 0 to 256. They're color, so they can display
00:04:33.28 you know, 0 to 256 in each of red, green or blue. But if
00:04:38.22 you're just looking in a single color, you only get 8 bit
00:04:40.29 resolution. So, before when I was showing these, you know,
00:04:44.23 16 bit grayscale traces on the bottom, we were scaling them such
00:04:49.07 that you know, the blackest pixel, the 0 pixel, was black.
00:04:53.04 And the brightest pixel, the 65000 pixel, was white.
00:04:56.20 But, what if instead we just scaled them so that the
00:05:00.18 actual intensity values are mapped directly onto the monitors?
00:05:03.23 So that 0 would be black and 255 would be white. Then you get
00:05:08.13 something that looks like this. You can see the small bit
00:05:11.03 depth images are only using the darkest gray levels on the monitor,
00:05:14.24 so they're very hard to resolve. The 8 bit one here is
00:05:18.03 obviously matched to the monitor, so we see the full
00:05:20.17 gamit going from black to white. And when we get to these larger
00:05:23.24 bit depths, the 4000 or 65000 gray levels, you now see that
00:05:28.10 all the dynamic range is compressed into this little tiny
00:05:30.17 bit at the bottom, and everything above 255 now just shows this
00:05:34.01 white. So this should hopefully bring up this point that
00:05:38.25 if you're looking at images that have a different bit depth than your
00:05:41.26 monitor, you need to do some kind of matching of the gray levels of
00:05:45.21 the image to the gray levels of your monitor. Or you're going
00:05:49.06 to either compress everything to either the dark shades or
00:05:52.02 saturate a tremendous amount in white, unless you happen to be
00:05:55.05 working with an 8 bit image. So you always have to think about
00:06:00.00 how you're going to do this mapping. And generally,
00:06:02.14 this is called a lookup table in most image analysis packages.
00:06:06.04 And the idea here is, you have the digital numbers that make up
00:06:09.18 your image on the bottom axis here, and on the side axis
00:06:13.18 here, you have the display intensity of your monitor. So here
00:06:17.09 we can take an 8 bit image, and we can control how we
00:06:20.10 map the intensities of the image to the intensities on the
00:06:22.25 display. And here we're just doing the obvious thing of
00:06:25.12 mapping with a straight line that goes from 0 to 255.
00:06:29.21 If instead, we use a much steeper line that now goes from
00:06:34.13 0 to maybe say 50, so we're saturating everything above
00:06:37.21 50 in white, you can see now the image gets very
00:06:39.22 bright and washed out. So this is for an 8 bit image, you can
00:06:45.02 equally image how you would do this if this bottom scale
00:06:48.03 instead of going from 0 to 255, went from 0 to 4000.
00:06:51.02 Or 0 to 65000. And for preparing images for presentation,
00:06:58.16 so say you're putting together a Powerpoint lecture like this one,
00:07:01.19 or preparing figures for a journal or publication elsewhere,
00:07:05.21 almost always you need 8 bit files there. Because computer
00:07:09.05 screens are 8 bit, and publishing workflow is designed around
00:07:12.09 8 bit files. And so if you have some image that has some larger
00:07:16.12 bit depth than 8 bit, if you have those 12 bit images that go from
00:07:19.01 0 to 4095, you need to think about how you're going to do this
00:07:23.01 mapping. And an important point here is that you lose information
00:07:26.01 in this process. Because we only have 255 values in our final
00:07:29.23 image, but we start with 4000 values in our initial image.
00:07:32.10 And so, for instance, all the values here in this mapping
00:07:35.12 where we're just mapping from the min to max, kind of
00:07:38.19 straight of line. All the values between 4080 and 4095
00:07:43.02 are going to end up mapped to 255 in their final image.
00:07:45.19 And so generally, this means you want to put off this
00:07:49.17 conversion as long as possible. You should do this as the last
00:07:51.24 step in your image analysis pipeline. If you do it too soon,
00:07:55.05 you lose this information, and that may come back to hurt you
00:07:58.23 in the end. When you're trying to bring out some detail
00:08:00.26 that was in the original image, but then got lost when you did
00:08:03.15 this conversion. So generally, you want to stay in the native bit
00:08:07.19 depth of your image, as long as you can, and then as the
00:08:10.06 final step, make an 8 bit image that you can send to the
00:08:12.18 publisher or put in your Powerpoint. So in this intensity
00:08:19.21 scaling here, in this look up table, you'll see a number of
00:08:23.07 terms that refer to how you can do the scaling. Right now
00:08:27.11 we're just talking about linear scaling, so you can define the
00:08:29.17 line as either the min and the max, that's how I've been talking about it.
00:08:33.09 But you'll often also see terms of contrast and brightness,
00:08:37.10 and so contrast refers to the slope of this line, so if you
00:08:40.09 make this line steeper, you'll get higher contrast. If you make
00:08:43.26 it shallower, you get lower contrast. And then brightness
00:08:47.26 is then the offset of this line along this plane in here.
00:08:51.21 So you'll see most software packages will let you set this
00:08:57.06 either by setting the min or the max or by setting the contrast
00:08:59.06 and brightness. And so here again is just this image, and
00:09:04.21 showing the effects of this brightness and contrast adjustment. And
00:09:07.25 now in addition to the image, we're showing the mapping down here
00:09:11.03 with this line. And also in this grayscale here, the pixel
00:09:15.14 intensity histogram. And so what this is, is just going through this
00:09:19.02 whole image and looking at every pixel and counting up
00:09:21.18 how many pixels there are at a given gray level. And so you
00:09:25.10 can see in this image, there is a big peak down here in the dark
00:09:28.20 pixels, that corresponds to the ground, and then another big peakhere of bright pixels that corresponds to the sky. And then
00:09:34.15 a smaller number of pixels of the stuff in between them.
00:09:38.22 And so you can see here, that in fact, this image doesn't
00:09:41.13 have anything that is completely at 255 or at 0.
00:09:44.02 That the ground is not totally black and the sky is not
00:09:47.15 totally white. And so, we can then adjust this scaling here
00:09:51.01 so that we're scaling just between the minimum value of the image
00:09:53.29 and the maximum value in the image. And you can see this makes
00:09:56.16 the ground totally black, and the sky in the brightest spots totally white.
00:09:59.10 And so this is called brightness/contrast adjustment, and
00:10:03.14 this is kind of a fundamental tool you'll need to adjust any time
00:10:07.19 you're looking at an image on a screen. And particularly if
00:10:10.17 you're looking at an image that has a bit depth that's
00:10:12.15 different from the screen you're using to display it on.
00:10:13.27 So, a corollary of this is that you want to be careful how your
00:10:20.11 software is scaling your image. So many software packages
00:10:23.16 will scale in different ways. And the two common defaults
00:10:27.12 are that it'll either scale to the full range, so here's a
00:10:29.29 16 bit image that goes from 0 to 65000. And the scaling
00:10:34.15 here is set that it's just mapping 0 to 0 on the monitor.
00:10:39.18 And 65000 to 255 on the monitor. And so you can see this image is
00:10:44.12 really dark and murky, because in fact, there aren't any pixels
00:10:46.20 here that are at the maximum intensity value of the 16 bit
00:10:49.07 format. There are no 65000 valued pixels. So an alternative
00:10:55.14 way to scale this is sort of how we did that image on the previous
00:10:58.16 screen, which is just to scale from the min to the max.
00:11:00.24 And so if we do that, now we scale between you know,
00:11:04.15 whatever that number is and 10000 or so, which represent the
00:11:07.20 darkest and brightest pixel in the image. And now you can see
00:11:11.15 you bring out this image much better and you can see this
00:11:14.08 is an image of a galaxy and some stars. And so it's very common
00:11:20.09 for software packages to do both of these things, either to
00:11:22.19 do default scaling the min and max, and the full range
00:11:25.26 of the data. Or to autoscale to the min and max in a particular
00:11:31.08 image you're looking at. And in particular, this autoscaling
00:11:34.18 is very nice because it makes it immediately easy to
00:11:37.21 see what's in your image. But if you load two different
00:11:40.19 images and autoscale them, they're not going to have
00:11:42.20 the same scaling. And so you can't directly compare
00:11:44.23 intensities between them by looking at them. So it's
00:11:48.16 important to be aware of what your software is doing.
00:11:50.25 Another option here is of course that we could instead of
00:11:55.00 scaling just to the minimum and maximum values in this
00:11:57.21 image, we could saturate some of the values in this image.
00:12:01.08 We could instead choose to scale from the minimum to
00:12:03.26 something below the maximum, and the result of that is
00:12:06.13 we set all these bright pixels here to 255 on the screen,
00:12:09.15 and in full white. These bright pixels here to 255. And
00:12:13.13 so we can't really resolve what's going on inside of those stars
00:12:16.01 now, but instead we can see very clearly this nice spiral
00:12:19.18 galaxy here, which was dimmer and not really well resolved
00:12:22.19 on our monitor before. Another option for scaling here is
00:12:28.29 to go beyond just simple linear scaling, and instead apply
00:12:32.01 some kind of non-linear scaling. And a very common one
00:12:34.27 you'll see is gamma correction. And this is basically an
00:12:39.20 exponential or logarithmic mapping of your data.
00:12:42.07 And gamma equals 1, it corresponds to linear scaling.
00:12:45.25 So this straight line in the middle here is the gamma equals
00:12:48.16 1 case. And then gamma values less than 1, basically
00:12:53.16 saturate things at high intensities, and really stretch out the
00:12:56.21 intensity differences between low intensity objects. And so you can
00:13:00.05 see here, that the slope of this curve is very steep for low intensities
00:13:03.24 on the input. And so we're making a lot of gray scale
00:13:08.10 distinctions between low intensity objects, but then as we get
00:13:11.24 to higher intensity objects, we're flattening those out
00:13:14.27 and not using as many gray scales to represent the really bright
00:13:18.05 objects. Conversely, a gamma above 1 does the opposite.
00:13:23.11 So it will compress all the dim stuff here to a small number of
00:13:27.06 gray scales, and stretch out the gray scales among the brighter
00:13:30.03 objects in the sample. And these are less commonly used
00:13:34.20 but they're very handy if you have images that have a
00:13:38.16 large dynamic range, where there's important differences
00:13:41.04 concentrated at one end of the intensity spectrum or the other.
00:13:44.09 And so just showing that applied to this galaxy image.
00:13:47.02 Here is gamma equals 1, in gamma is above 1, you can see here
00:13:52.04 that it really makes it hard to see the dim stuff in the object.
00:13:54.27 But here, gamma below 1 helps you bring out those dim
00:13:58.25 background pixels that make up that spiral galaxy
00:14:00.21 that otherwise would go missing.
00:14:04.17 Okay, so with all these things we can do to our images,
00:14:07.10 what's acceptable? What's sort of the scientific standard here
00:14:11.12 for what you can do in a published paper?
00:14:15.05 And the Journal of Cell Biology has put a lot of work into
00:14:17.23 this, and they have some very nice guidelines which you can
00:14:20.04 find at that these websites here. But to summarize them briefly,
00:14:24.10 the general idea is that brightness and contrast adjustments are
00:14:27.12 okay. So long as you do them over the whole image, and you
00:14:30.26 don't obscure or eliminate background. So you don't
00:14:33.00 want to set your zero levels so high that you suppress
00:14:37.27 real background signal in your images. And you also don't want
00:14:42.12 to treat different objects in your figure differently, or
00:14:45.27 different objects in the same field of view differently.
00:14:48.11 They require that nonlinear adjustments, like gamma
00:14:51.19 corrections, be disclosed. And importantly, you can't cut and
00:14:55.18 paste regions within an image. So you can't make a composite of
00:14:58.16 multiple images, where it's not obvious by white bars between the
00:15:01.24 images that say you're making a composite. They don't
00:15:03.15 want you to put things together that weren't actually together in real life.
00:15:08.10 And finally, and a particularly critical point is, to make any
00:15:12.10 real comparisons in your image, you have to treat your control
00:15:14.23 and experimental data identically. So you want to show them
00:15:17.13 using the same scaling and the same gamma corrections,
00:15:19.16 if you've done a gamma correction and so on.
00:15:21.13 Okay, so that's sort of the basics of how to display
00:15:27.02 grayscale images. And now I want to talk a little bit about using color.
00:15:29.29 So, so far we've completely restricted ourselves to using
00:15:34.16 black and white grayscale image displays. But there's no reason
00:15:39.01 you can't use color. And so here's an example of just a 2 bit
00:15:43.13 image. It's got four different values, 0, 1, 2, and 3.
00:15:47.08 Here are the pixel values, and here is what this thing looks like.
00:15:49.27 It's a little bright spot. And so we can display it in gray
00:15:54.29 scale the way that we've been doing all along right here.
00:15:56.25 But we can also map different gray values to different colors
00:16:01.16 on the display. And so shown here is a color map where we're using
00:16:06.09 black and shades of green to display this object. And then that
00:16:09.13 false colors it green. But we don't need to stop there, we can
00:16:12.14 use bizarro color scales like this one, which map 0 and 3,
00:16:16.13 the darkest and brightest thing, both to red. And you can
00:16:20.09 see what it looks like there. You can use, you know,
00:16:23.04 whatever color scale you want basically. And that sometimes
00:16:26.23 is a good way to bring out small differences in your image.
00:16:29.12 So here's an example looking at kind of a fairly uniform
00:16:33.04 grayscale image. This happens to be a brightness or a
00:16:38.03 flat field correction for a microscope, where we're just looking
00:16:40.29 at the uniform illumination on a microscope. And we've scaled
00:16:45.08 this to the min and the max in the image. So here's the
00:16:47.12 gray scales, they run from 7000ish to 8 or 9000 here.
00:16:52.05 And you can see a lot of structure in this, but that becomes a lot clearer
00:16:57.02 if we now false color this, rather than just in shades of
00:16:59.18 gray, and this color map, this heat map, goes from blue
00:17:02.00 for dark objects all the way up to dark red for bright objects.
00:17:05.17 And you can see when we do that, now we can make out
00:17:10.06 fine details in this image more easily. And it also makes
00:17:13.18 it easier to map the shades of color here to intensities
00:17:18.08 I'm looking up at this color scale here, just because it's easier
00:17:21.08 to distinguish differences in color than between
00:17:24.17 two gray scales. We can also use this procedure to make
00:17:30.06 false colored images. So here's an example of taking three
00:17:33.12 images. These are three color stained tissue culture cells.
00:17:36.16 Stained for nuclei up here, mitochondria here, and actin
00:17:40.16 down here. And shown on the left here is just the grayscale
00:17:45.00 mappings, and on the right is applying the particular color
00:17:47.27 scales here from these bars in the center to that image.
00:17:50.16 So we've mapped the nuclei in cyan, the mitochondria in yellow,
00:17:53.29 and the actin in this sort of magenta color. And if we now
00:17:59.14 put all three of these images together and overlay them,
00:18:01.26 we get this very pretty funky colored image.
00:18:07.25 And so that brings me to color images. So most microscopy
00:18:13.24 and especially most fluorescence microscopy, is done with
00:18:15.24 grayscale images. Just because our cameras are monochrome,
00:18:20.25 and that's usually what we detect. But color images are
00:18:23.12 also obviously quite important. Particularly for making figures for
00:18:26.23 publication, as well as for non microscopy applications or
00:18:31.03 microscopy applications where you're looking at H&E
00:18:34.09 stained specimens or other pathology type specimens,
00:18:36.26 where you want a true colored image of your sample.
00:18:39.10 So a color image is nothing more than three gray scale images,
00:18:43.08 one image each for the red channel, the green channel,
00:18:47.24 and the blue channel. And these like any other images can have
00:18:51.16 bit depths, more or less standard is they're either 8 or 16
00:18:54.27 bits per channel. So if we look at just a color image here,
00:18:59.27 this can be decomposed to these three images of red,
00:19:03.12 green, and blue. And then each one of those is really just
00:19:06.28 a monochrome image here with the appropriate look up table
00:19:11.11 applied to it. So you know, red look up table for the red,
00:19:14.21 green lookup table for the green, and a blue lookup table for
00:19:16.21 the blue. And these can be stored in a number of different
00:19:22.05 ways. You can either store for each particular pixel,
00:19:25.24 either the red value or the green value or the blue value,
00:19:27.14 and then for the next pixel, the red value, the green value,
00:19:29.11 the blue value. Or you can really just store these three separate
00:19:32.17 images as three planes. A red plane, which has all the red
00:19:36.05 pixels, a green plane that has all the green pixels,
00:19:38.04 and a blue plane that has all the blue pixels. You don't
00:19:41.22 really need to worry about this most of the time, because your
00:19:44.14 software should take care of the loading and properly sorting
00:19:47.17 out how to display the color information in a color image.
00:19:51.02 In microscopy, it's common to have more complicated
00:19:56.26 image formats than just black or white or color.
00:19:59.11 You often have stacks or sequences of images, say if you've
00:20:03.13 taken a movie. Or maybe you've taken a through focus section here
00:20:07.04 and a different set of z-planes here in your sample.
00:20:08.29 Or other variables, more colors than just red, green, and
00:20:13.04 blue. You could do five or six wavelengths in an image.
00:20:15.10 You could do multiple stage positions, say in a 96 well
00:20:18.22 plate. And so that would give you this 3-dimensional stack
00:20:22.22 here where you've got a 2D image for each of these
00:20:25.18 planes. And in fact, in microscopy now, it's very common
00:20:28.17 to do not just stacks but what people call hyperstacks or
00:20:32.13 multidimensional images, where maybe you've got a
00:20:34.28 time series. And at each timepoint, you've got z-positions,
00:20:37.27 and at each z-position you've got multiple colors. So you've got
00:20:40.05 a four dimensional image stack. And so, how to store
00:20:47.14 and handle all this data becomes a sort of issue.
00:20:49.24 And so that's what I want to talk about now, file formats
00:20:54.08 and how you store it and work with this data. And in particular,
00:20:59.00 images can get quite big. Well, not the images themselves,
00:21:02.22 as much as the whole data set. A very common camera used in
00:21:06.08 microscopy has these dimensions, it's a 1392x1040
00:21:10.02 pixel camera, so it's about 1.4 - 1.5 megapixels. And it
00:21:15.00 records 14 bit images, so you need 2 bytes to store the
00:21:19.08 data. So that requires 2.8 megabytes just to store a single
00:21:22.13 image from that camera. So now, image you're doing some
00:21:27.01 complicated multidimensional experiment, where you're
00:21:29.10 acquiring three channels. At each position, you're acquiring
00:21:33.13 15 images, and then you're doing this whole thing
00:21:36.19 for 200 time points. So at each time point, you're going
00:21:39.04 to record your 15 z-positions, and then at each z-position,
00:21:42.05 you record your 3 channels. So if you multiply this all out,
00:21:45.14 you've got your 2.8 megabytes times the 3 channels, times the
00:21:49.11 15 z-positions, times the 200 time points, and that works
00:21:52.08 out to a 25 gigabyte data set. And this is not a particularly
00:21:56.14 complicated microscopy experiment. You might often see this
00:22:00.16 combined now in doing multiple positions in your sample.
00:22:03.06 And this just gets bigger and bigger. So data sets that are
00:22:07.18 10, 20, or more gigabytes are not uncommon. I know
00:22:10.03 people who routinely record 200 gigabyte data sets in
00:22:13.04 an evening. And so obviously, if you're storing data sets this large
00:22:17.29 you need to really think about how best to store them. And
00:22:20.24 particularly, how to keep track of all the information of where
00:22:23.04 each one of these images came from. You know, this now has
00:22:26.20 thousands of images and you need to know the z-position
00:22:29.21 and time point and the channel that each image corresponds to.
00:22:31.21 And this also brings up this issue of compression. Because
00:22:37.10 you'd like to not store 25 gigabytes if you don't have to.
00:22:41.02 So, there are two kinds of compression used for images
00:22:45.13 or used for things in general. And there's lossless compression
00:22:50.10 and lossy compression. And knowing the difference between these is
00:22:53.28 really critical. So lossless compression is exactly as it sounds like,
00:22:58.15 it preserves all of the information in your original image.
00:23:01.12 It compresses your image by removing redundant information
00:23:04.18 that can be restored after the fact. So this means you can always
00:23:07.15 get back your original image, your original raw data
00:23:09.22 from this image, despite the fact that it's been compressed
00:23:12.05 and takes up less space. Most compression you're probably
00:23:16.20 familiar with is not lossless. But it's lossy compression.
00:23:20.05 And so this is the kind of compression that used for
00:23:22.24 sort of consumer cameras for standard imaging with
00:23:29.15 your iPhone or your digital camera. And in order to make
00:23:34.08 images smaller here, what this does is it throws out
00:23:37.03 image data that's not visually obvious. So it removes
00:23:42.14 small details in the image that you can't really see.
00:23:45.27 But there is real data there that's being thrown away.
00:23:49.16 And in fact, if you turn up the compression on one of these lossy
00:23:54.23 compression algorithms, eventually you start to see
00:23:56.16 the artifact it's introducing. But most of the time, people who
00:24:01.14 write these algorithms try to design it so that the way it
00:24:04.11 throws out information is not really obvious to your eye,
00:24:06.17 but it is definitely throwing out information nonetheless.
00:24:09.10 And that means you can't get back your raw data here.
00:24:11.25 Once you've gone through one of these compression algorithms
00:24:13.25 that's lossy, JPEG is a very common one, you've lost
00:24:18.15 information about your image and you can't get it back.
00:24:21.01 And so that means for doing scientific work, these
00:24:25.13 are really terrible ideas. Because you've corrupted your
00:24:29.26 raw data, and you may be introducing artifactual stuff in
00:24:34.23 there. Introducing artifacts into your data that may affect
00:24:39.03 your conclusions. And there's a really great write up
00:24:42.24 on this website here about data compression for
00:24:46.20 both images and video. But the take home message is for
00:24:50.00 any scientific work, you want to be using lossless compression
00:24:53.06 and not lossy compression. So I want to just talk about
00:24:58.13 file formats a little bit. One of the most common file formats
00:25:02.17 you'll see used in imaging is TIFF, it's a fairly old file
00:25:05.25 format. But it supports both 8 and 16 bit images, which is nice
00:25:10.04 because we work with both. It supports lossless compression,
00:25:13.22 which is also nice because we don't want to use lossy
00:25:16.05 compression. And it supports grayscale or RGB. The downside
00:25:20.04 to it is that in general, TIFF store only a single image or
00:25:24.08 a single color image. So that means in our example where we're
00:25:27.05 taking three channels and 15 z-stacks, and 200 timepoints,
00:25:31.20 we're going to need thousands of TIFFs to keep track of all those
00:25:35.16 images. And so that's not so nice. There are other file formats, there's
00:25:40.29 a new variant of JPEG, JPEG 2000, which you don't see
00:25:43.03 that much, which is lossless and quite nice. And then
00:25:46.29 basically to deal with this issue of keeping track of these
00:25:49.28 thousands of images, most of the microscopy companies
00:25:53.15 have come up with their own custom format, so you see
00:25:55.19 things like IDS and ND2, that's a Nikon format, ZVI and LSM
00:25:59.24 are Zeiss formats and so on. and these are generally
00:26:04.05 pretty nice. They're lossless and they support the full
00:26:06.16 bit depth of your system. Almost always those custom formats
00:26:10.28 will support multidimensional images, so you can save that
00:26:14.17 entire three channel by 15 z-slice by 200 time point data
00:26:20.18 set as a single file, as a single large file, but a single
00:26:23.27 file. And these custom formats will also often keep track of
00:26:28.11 the metadata of your experiment. It'll keep track of
00:26:30.22 like how your microscope was set up and what the z-spacing
00:26:34.21 was in your z-stack. And what the different channels you used were.
00:26:36.29 What the time interval in your timelapse was.
00:26:40.09 And so those are very nice options, but the downside
00:26:43.02 to them is that because they're custom, they're not always
00:26:45.14 portable. So you certainly won't be able to open these on
00:26:48.21 something like Photoshop, you can always open them in the
00:26:52.13 manufacturer's software and generally you can find plugins
00:26:54.29 that open them in things like ImageJ. But if you want to get them
00:26:58.00 into something say, a custom program for doing image analysis,
00:27:00.04 you'll probably have to convert them to a TIFF stack
00:27:03.02 first. So a whole bunch of TIFFs and then load them to
00:27:06.14 your program. And then there's a whole bunch of file formats
00:27:09.26 you probably want to avoid. And those are things like JPEG,
00:27:13.13 because I said it's lossy, this old GIF format, or BMP, and all
00:27:19.08 these formats that were designed more for ease of
00:27:22.13 use or computer simplicity than for really preserving data.
00:27:27.21 So many of these are restricted to 8 bit only, and some
00:27:31.25 like JPEG also use lossy compression automatically. And so
00:27:35.08 in general, they require corrupting your data in some way you
00:27:38.04 probably don't want to do. They may be fine for putting
00:27:41.16 pictures on your website, but you don't want to use them in your
00:27:44.11 scientific data processing chain. So finally I just wanted to
00:27:49.19 say a few words about the software tools that are out there for
00:27:52.15 doing image analysis and working with images. There's a really large
00:27:56.18 number of them, so I'm not going to go into any detail here,
00:27:59.22 but just to sort of summarize what's out there.
00:28:02.05 So there are a large number of programs that will do both
00:28:04.24 acquisition, meaning they'll control your microscope, and
00:28:07.23 do data analysis. And so there's a large number of commercial
00:28:11.18 ones, NIS-Elements is Nikon's software, AxioVision
00:28:15.00 is Zeiss's software. MetaMorph is a software package that's been
00:28:19.01 around a long time that's microscope independent, and will control
00:28:23.09 many different microscopes. Slidebook is a similar program
00:28:27.09 to MetaMorph, in that way. And then Micro-Manager
00:28:29.28 is one I want to touch on, because it's developed at UCSF.
00:28:32.17 And it's free and open source, and it's basically a
00:28:37.00 plugin that lives inside of ImageJ and will control microscopes.
00:28:40.14 And then you can use the ImageJ functions for data analysis.
00:28:43.25 So all these programs will let you both control your microscope
00:28:46.24 and do many different data analysis tasks. Then there's a bunch
00:28:51.00 of presentation tools, like Photoshop which you're probably familiar
00:28:54.09 with or the freeware open source version of this, Gimp.
00:28:57.14 Which are designed not really for doing scientific data
00:29:00.21 analysis, but for making attractive figures and putting
00:29:03.18 data together so that you can display it in a figure or
00:29:06.10 in a presentation. And then there's a bunch of sort of dedicated
00:29:11.21 image analysis packages, and these include things
00:29:13.24 like Matlab, which is really a general purpose programming
00:29:16.27 language in many ways, but it has very nice tools for doing
00:29:19.06 image analysis. ImageJ, which is this open source, free
00:29:25.07 tool that has many different image analysis routines built
00:29:29.06 into it that you can use for doing image analysis. Imaris
00:29:33.09 which is a commercial package, which is really optimized for doing
00:29:36.03 high end 3D visualization. CellProfiler, which is another free open source
00:29:41.25 tool that's optimized for doing high throughput screening
00:29:46.05 analysis, so analyzing thousands or tens of thousands of
00:29:49.01 images automatically. And there are many others I haven't
00:29:52.10 mentioned here, which are sometimes specialized for particular
00:29:55.15 tasks, or just a little less commonly used than these.
00:29:59.24 So that's all I'm going to say today, and I just want to thank
00:30:04.12 Nico Stuurman, who provided a lot of the slides and the
00:30:07.21 outline for this talk. Thank you.

This Talk

Speaker: Kurt Thorn

Audience:

Researcher

Recorded: April 2012

Talk Overview

Digital images are collections of measurements of photon flux. To display, manipulate, store and make measurements of digital images, you need a basic understanding as provided in this lecture.

Questions

What is a binary image and what is a 8-bit image?
1. A binary image has 2 gray-scale values (black and white) and an 8-bit image has 8 gray-scale values
2. Every digital image is a binary image and an 8-bit image has 8 gray-scale values
3. A binary image has 2 gray-scale values (black and white) and an 8-bit image has 256 gray-scale values
4. Every digital image is a binary image and an 8-bit image has 256 gray-scale values
To display a 12-bit image on an 8-bit monitor, you have to
1. Change the monitor’s brightness and contrast
2. Map the values of the 12-bit image by changing brightness and contrast and possibly gamma such that the important features of the image are visible
3. Rely on the auto-adjust function of the computer program and never change anything
4. Buy a 12-bit monitor
A histogram of a digital image generally refers to
1. A plot of the total number of pixels
2. A plot of intensities on the y axis and the number of such pixels on the x axis
3. A plot of intensities on the x axis and the number of such pixels on the y axis
4. A plot of the total number of intensities
A digital image of cells expressing GFP is always green, True or False?
To store your images such that they can be used for subsequent analysis you can best use the following file format:
1. TIFF
2. GIF
3. JPEG
4. BMP

Answers

View Answers

Speaker Bio

Kurt Thorn

Kurt Thorn is an Assistant Professor of Biochemistry and Biophysics at UCSF and Director of the Nikon Imaging Center – a facility that provides cutting edge light microscopy equipment to UCSF researchers. Kurt can be followed on his blog at http://nic.ucsf.edu/blog/. Continue Reading