Monday, November 5, 2012

Annals of underwhelming papers: Channelrhodopsinning

While I am no Boyden-level expert in channelrhodopsin, I have photostimulated neurons in the olfactory bulb, and now am trying to use the delightfully named PINP technique to record from specific layers of cortex. Given this rudimentary experience, I am increasingly frustrated by the sloppiness of optogenetic research in good journals. Today I would like to highlight four articles that used channelrhodopsin sub-optimally. (To be 100% clear, I am not commenting on the quality of these papers as a whole, or the researchers who conducted the experiments. I am merely saying that I think the optogenetic experiments could have been done better).

Two Pillars of Channelrhodopsin Photostimulation

In the channelrhodopsin experiments I have done so far, I have identified two keys to channelrhodopsin use, which I have dubbed the Two Pillars of Channelrhodopsin Photostimulation.*

The first pillar is to always perform negative controls. Light is energy, which means light can cause heat. Light can heat tissue, but Michale Fee even thinks that metal electrodes can heat up if stimulated long enough. For example, in a set of experiments I did last fall, I photostimulated neuromodulatory neurons while recording from the olfactory bulb, and saw some interesting changes in activity. When I performed the histology, I found out that the infection had not taken, and what I was witnessing was a heat-induced artifact. To control for heat during photostimulaiton, you must do two things: first, reduce your stimulation power and time as much as possible to minimize the risk of heat artifacts; and second, always, ALWAYS perform negative controls in non-channelrhodopsin animals.

* Back in my two-photon days, it was easy to put your slice under the microscope, turn to the computer, and not see an image. 90% of the time this was due to a few routine mistakes, so whenever I taught people how to use the microscope, I invented a Buddhist-style "Four Pillars of Two-photon Imaging." (Move the filter wheel, close the fluorescence shutter, turn down the fluorescence lamp, and switch the imaging mirror.)

The second pillar is to pulse your light. Channelrhodopsin is a non-specific cation channel. This means that when the channel initially opens, the increase in conductance causes a large depolarization. However, if you keep the channel open, you have no idea what's going on. Some neurons will continue to happily fire action potentials, while others will go silent. You could even, theoretically, get shunting inhibition from the increased conductance. To avoid these problems, and have a better idea about what your stimulated neurons are doing, you should pulse your photostimulation for 1-10 ms, and stimulate at 1-50Hz (or even higher if you have a new fancy channelrhodopsin). Pulsed light stimulation also helps avoid heat artifacts.

Given these pillars, I'd like to highlight four papers in high profile journals from big name labs that did not use them. Hopefully I don't stick my foot in my mouth.

Poulet et. al., 2012

From the Petersen lab at EPFL, this paper followed neither pillar. They recorded intracellulary from neurons in barrel cortex while photostimulating thalamic neurons expressing ChR2. When they stimulated the thalamus, the used a 5 second ramped light pulse.

Intracellular recording from barrel cortex during photostimulation of the thalamus. Photostimulation caused a change in the frequency of membrane fluctuations.
From Figure 3, Poulet et. al., 2012.
In the supplementary information, they tried to justify... something by showing that a stepped light pulse caused whisker deflections, while ramped light pulses did not cause whisker deflections (see below). To me, this is just another argument in favour of short, pulsed stimulation. With the step pulse, the whisker activity was at the beginning of the pulse, when you know the photostimulation is causing spikes; the lack of whisking afterwards shows you don't know what's happening during the last four seconds of the pulse. Furthermore, the ramped pulse elicits no whisking activity, showing you don't know what the heck that protocol is doing.
(left) Square pulse photostimulation causes whisker deflections (individual trials, green traces), primarily at stimulus onset. (right) Ramped photostimulation causes no whisker deflections.
From Supplemental Figure 6, Poulet et. al., 2012.
I don't know squat about barrel cortex or the thalamus, but this is a flabbergasting experiment: their light stimulation is too long, and may cause heating; they don't have any stimulus control; and they never performed photostimulation in a non-ChR2 animal.

Tan et. al., 2012

From the Luscher lab next door, this paper has a couple problems. They recorded in vivo from the VTA of GAD-cre mice infected with floxed-ChR2 (viz. GABAergic ChR2 neurons). To identify whether neurons were GABAergic, they used 1 second light pulses. This is a minor sin, since the neurons are fast spiking, and they infected GABAergic neurons, making non-specific excitatory activity unlikely. However, they really should have PINPed it.

Extracellular recording in vivo in GAD-ChR2 mice. Photostimulation excites GABAergic neurons.
From Figure 1, Tan et. al., 2012.
More troubling, they performed a conditioning paradigm, where "the laser was continuously activated when mice entered the conditioned chamber for a maximum duration of 30 seconds to avoid any overheating of the brain structures." In my opinion, a better way to avoid heat artifacts would have been pulsing the laser. They did perform negative controls in non-expressing animals.

Atallah et. al., 2012

This paper from the HHMI Scanziani lab looked at how inhibitory neuron activity shapes pyramidal neuron tuning properties. They expressed ChR2 in parvalbumin cells, and recorded in vivo using loose-patch. They then stimulated the cells using one second light pulses (blue bar), in combination with visual stimulation (grey area). Like Tan above, this is a minor sin, since they recorded from the cells they photostimulated, and know what activity they are inducing. In the supplement, they also provide negative controls.

(Left) Image of a tdTomato-ChR2 expressing parvalbumin cell, recorded via loose-patch. (Right) Raster plot and PSTH from the cell in response to visual stimulation (grey area). The activity was higher for a combination of light (blue bar) and visual stimulation.
From Figure 2, Atallah et. al., 2012.
Kravitz et. al., 2012

From Young Investigator Kreitzer's lab, this paper looks at how the direct and indirect pathways in the striatum influenced learning.** To ensure that they were stimulating the cell type they intended, they used a chronic in vivo optrode to record spikes in response to light stimulation. For a veneer of control, they used different light intensities, ranging from 0.1-3mW, but used constant 1 second light pulses.
PSTH of a neuron in response to constant 1 second light pulse.
From Supplemental Figure 1, Kravitz et. al., 2012.
In their words, "Neurons were classified as ChR2 expressing if they exhibited, within 40 ms of the laser onset, a firing rate more than threefold greater than the s.d. of the 1 s preceding the laser pulse." This strikes me as a crap criterion, since if there is any excitatory feedback in the network, it would be active within 40 ms. For example, the gradual increase in firing rate for this neuron looks odd. I asked the Luscher lab about connectivity in the striatum, and they said that the direct and indirect pathways inhibit each other, so it is unlikely in this case that Kravitz is recording the wrong neuron type. In any case, this is sloppy.

** Which Cre mice did they use? "Bacterial artificial chromosome (BAC) transgenic mouse lines that express Cre recombinase under control of the dopamine D1 receptor and A2A receptor regulatory elements were obtained from GENSAT." Is it too much to ask them to name which of the nine D1A or three A2a lines they used? There can be variability between lines which nominally target the same populations.

Maybe I'm just being a pedantic curmudgeon. Maybe it's my background in cellular neuroscience where you always have to perform negative controls. But I think we should have higher standards for channelrhodopsin experiments. If these papers in high tier journals from respected labs (including Deisseroth on one paper!) can get away with sloppy channelrhodopsin work, what's getting through in other journals? If people simply followed the Two Pillars of Channelrhodopsin Photostimulation, their results would be cleaner and more reproducible, and get less guff from reviewers like me.


Atallah BV, Bruns W, Carandini M, & Scanziani M (2012). Parvalbumin-expressing interneurons linearly transform cortical responses to visual stimuli. Neuron, 73 (1), 159-70 PMID: 22243754

Kravitz AV, Tye LD, & Kreitzer AC (2012). Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nature neuroscience, 15 (6), 816-8 PMID: 22544310

Poulet, J., Fernandez, L., Crochet, S., & Petersen, C. (2012). Thalamic control of cortical states Nature Neuroscience, 15 (3), 370-372 DOI: 10.1038/nn.3035

Tan KR, Yvon C, Turiault M, Mirzabekov JJ, Doehner J, Labouèbe G, Deisseroth K, Tye KM, & Lüscher C (2012). GABA neurons of the VTA drive conditioned place aversion. Neuron, 73 (6), 1173-83 PMID: 22445344

Thursday, November 1, 2012

PINPing ain't easy

With my olfaction research ending at Zeno speed, I've started pilot experiments for everyone's favourite sensory modality, taste! The idea is to record from each layer of taste (insular) cortex, and compare the "taste receptive fields" of neurons from each layer. To identify the layer I record from, I'm using channelrhodopsin-aided cell identification: I record extracellulary from random neurons in cortex; the mouse I'm recording expresses ChR2 in a specific cortical layer; and then I identify which layer of cortex I'm recording from by photostimulating and recording evoked spikes. To get ChR2 expression in specific cortical layers, I'm using transgenic mouse lines that express Cre in specific cortical layers, and crossing those Cre mice with floxed-Channelrhodopsin (all lines are from MMRRC, and the names are available on request).


This technique has been dubbed PINP (Photostimulation-assisted Identification of Neuronal Populations) by the Zador lab (you can blame them for the label). In the original PINP paper, to get cell-specific expression of ChR2, Lima and colleagues used parvalbumin-Cre mice, and injected floxed-ChR2 (AAV-LSL-ChR2-YFP) into A1 of auditory cortex. They then stuck a tungsten electrode into A1 (in vivo), and recorded spikes. Some units reliably responded to light stimulation (panel B, below), while other units did not (panel C, below); the ones that responded to light were 
presumably parvalbumin-expressing neurons.

From Lima et. al., 2009: "Figure 4. In vivo photostimulation of parvalbumin expressing auditory cortex neurons. (A) PV expressing neurons in the mouse auditory cortex, labeled with the binary Cre-AAV system, were tagged with ChR2 (green). (B) Spike rasters of a well isolated single unit that responded to light activation in the mouse auditory cortex. Light was on from 0 to 10 ms. (C) Reliability of light-evoked responses in all the cells recorded in the mouse auditory cortex. Reliability was computed as the fraction of trials in which the firing rate within the 40 ms after the start of the light pulse was greater than within the 40 ms immediately preceding the light pulse. (D) Action potentials originated from ChR2-expressing neurons were narrower than spikes originated from the rest of the population (green - ChR2 positive, gray – unlabeled cells)."
One thing that surprises me about this experiment is that the spikes were delayed by ~2.8 +/- 1 ms. In other channelrhodopsin papers, people are able to get strong channelrhodopsin effects with light pulses of only one or two milliseconds. For example, Smear et. al. expressed ChR2 in olfactory epithelial cells, and mice were able to detect fine changes in photostimulation latency using 1 ms pulses. For another example, Atasoy et. al. were able to detect strong IPSCs in brain slices following a 1 ms pulse (Fig. 3b). If Lima and Zador had used 1 ms pulses, they may not have been able to evoke spikes. Given the wide variability in cell sizes, conductances, and excitability, when performing PINP, it is probably important to calibrate the duration of the light pulses.

Other old school PINPs

Almost all of the papers that cite Lima are reviews about the potential of optogenetics (and there are a LOT of reviews about optogenetics), but I also found a few papers that actually used the PINP technique. The first is from the lab of Young Investigator Rui Costa. Xin and Costa recorded from the substantia nigra, and were interested in differentiating between dopaminergic and GABAergic neurons. In addition to well-established criteria like spike width and firing rate, they used PINP to identify TH-ChR2 neurons (generated by infecting AAV-ChR2 in TH-cre mice). They implanted tungsten microwires into the substantia nigra, and stimulated the area with 10 ms light pulses.

Raster plots and PSTHs from putative TH-ChR2 neurons following photostimulation at t=0 ms. Recorded using tungsten microwires, and photostimulation pulses of 10ms.
From supplemental figure 3 from Xin and Costa, 2010.
During photostimulation, they saw an increase in firing rate in the putative TH-ChR2 neurons. To be honest, I find these responses underwhelming due to the jitter. In the example above from Lima and Zador, the evoked spikes were delayed, but extremely precise. In comparison, this simply looks like an increase in firing rate. I know nothing about the connectivity of the substantia nigra, but if you told me that the above raster plots were recorded from a neuron downstream of TH-ChR2 neurons, I would not argue with you.

The other example PINP I found comes from the lab of Naoshige Uchida, a former olfaction researcher. Cohen and Uchida were interested in how dopaminergic and GABAergic neurons in VTA encoded reward. They targeted dopaminergic neurons using DAT-cre, and GABAergic neurons using Vgat-cre, and expressed ChR2 via AAV. They recorded extracellularly using twisted wire tetrodes, and stimulated using 5 ms laser pulses.

a. Voltage trace of light evoked activity. b. Photostimulation at 20 and 50 Hz evokes reliable and precise spikes. c. (left) Plot of photostimulat-ability ("light-evoked energy") versus the correlation between spontaneous and evoked waveforms for individual units. Only those cells that had light-evoked spikes, and well correlated waveforms were considered to be ChR2-expressing. (right) Example waveforms for different units shown on the left.
From Fig. 3, Cohen et. al., 2012
Following photostimulation, they were recorded precisely timed spikes (panels a & b, above), at up to 50Hz. To ensure the light-evoked units were the same as the natural units, they performed a control I quite liked: they calculated the correlation coeffecient between the waveform of the photostimulated spikes and the natural spikes (panel c, right). Only those units that were both light-activated and maintained spike shape were considered to be TH-ChR2 expressing (panel c, left, filled blue dots). The precision of these spikes casts further doubt in my mind on those observed by Xin and Costa.

First PINP steps

I have started to try to perform PINP myself, and am using this post in part to structure my thinking about what is happening. Like all the previous papers, I'm using a Cre-lox system to get layer specific expression of ChR2. Unlike the previous papers, however, I'm using a transgenic floxed-ChR2 mouse to get expression.

Layer specific expression of YFP-ChR2 in somatosensory and taste cortex (blue box). Sagittal section taken at ~ -1.2mm from Bregma. Please pardon the overexposure.
As a pilot experiment, I anesthetized a mouse, and recorded from sensory cortex (I would have recorded from gustatory cortex, but had not stereotaxically head-posted the mouse, and decided not to risk an electrode recording by the edge of the skull). For recording, I used a Silicon optrode from NeuroNexusTech (A1x32-Poly3-5mm-25s-177-OA32). For my recordings in the olfactory bulb, I used electrodes that had an area of 312 um2. However, NeuroNexus does not make optrodes with electrode sizes that large (perhaps due to the photoelectric effect), and the electrode size for this experiment was 177 um2. In the olfactory bulb, using the 177 um2 electrodes, I recorded fewer spikes than with 312 um2 electrodes. Whether due to electrode size, the fact that the cortex is generally quiet, or due to the anesthesia, I didn't record any spikes in somatosensory cortex with the optrode.

Despite not recording any spontaneous spikes, I still tried photostimulating. The optrode consists of a single shank with electrodes forming two columns 200-300 um long; the optic fibre starts 200um above the top electrode. While doing this experiment, I neglected these measurements, and made a mistake. Somatosensory cortex is ~1mm deep, and the somas of the ChR2-expressing neurons are in the deepest layer. However, I only penetrated the optrode 500-600um, so I would not have been able to record evoked spikes.

Given the caveat that my electrodes were not in the layer of interest, I was able to record light-evoked LFPs. For the initial stimulations, I saw strong photoelectric artifacts at the beginning and end of the light pulses because the laser power was too high (panel A, below). The photoelectric artifact was easily removed by simply reducing the laser power.

LFPs recorded from light stimulation. A. (top) Light pulse. (bottom) LFP recorded. At high laser power, there is a photoelectric artifact at the beginning and end of light pulses. B. At medium laser power, there is no longer a photoelectric artifact. A 10 ms light pulse evokes a 2 ms LFP deflection. C.  A 1 ms light pulse evokes only a 1 ms LFP deflection.
With a more reasonable laser power, I was able to record LFP activity on every electrode I recorded. Even for longer light pulses, the LFP activity was fairly short. Whether this reflects a single evoked action potential in the deep layers, or something else is uncertain. To see if I could get the LFP activity with short laser pulses, I reduced the pulse duration to 1 ms, half the LFP duration. This in turn reduced the duration of the LFP activity. So there is some dynamic range in stimulation between 0 and 2 ms.

As the title of this post reflects, PINPing ain't easy. There are a few improvements I need to make before graduating to recording from taste cortex of (trained) awake mice. First, I need to penetrate the damn electrode to the layer of interest. This will be easy in somatosensory cortex, but may be complicated in insular cortex given that the layers are parallel to electrode penetration. Furthermore, I'm performing these experiments on mice head-posted 1-2 weeks earlier, which will complicate stereotaxy. Second, I need to be able to record spikes. All of the papers above recorded using tungsten wires, and got clean units. I'm concerned that the electrode sizes we are using are simply not large enough to record large number of clean units. Hopefully the difficulty of recording spontaneous spikes was due to the anesthesia and layers recorded. Finally, I am somewhat concerned that the transgenic expression is simply too high. If photostimulation activates all the neurons in a layer, will I be able to record single units? Or will the coactivation muddle any recordings? And will the strong LFP activity make isolating spikes difficult?

I have two priorities right now. First, I need to solidify the behavioural paradigm, and see how many training sessions it takes for mice to acquire the protocol, and measure how many trials I can expect out of a mouse before it is sated. And second, I need to refine PINP, and make sure that I can actually record from ChR2-identified neurons.


Cohen JY, Haesler S, Vong L, Lowell BB, & Uchida N (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature, 482 (7383), 85-8 PMID: 22258508

Jin X, & Costa RM (2010). Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature, 466 (7305), 457-62 PMID: 20651684

Lima SQ, Hromádka T, Znamenskiy P, & Zador AM (2009). PINP: a new method of tagging neuronal populations for identification during in vivo electrophysiological recording. PloS one, 4 (7) PMID: 19584920

Thursday, September 27, 2012

Concentration coding in the awake mouse olfactory bulb

It's expansion time here at the paper trail (viz., I'm preparing a manuscript), so why not use the trimmings for a blog post, and write even more?

Concentration coding in insects

When I walk into our behaviour room and smell something malodorous, I can identify the smell as mouse shit, even though the smell is less intense than when I'm working closely with a mouse (pardon the example, but mouse shit is salient for me now that I'm doing a bit of behaviour). The fact that you can identify the odor across different concentrations gives the odor it's "odor identity." Yet, in the brain, faint mouse shit and strong mouse shit evoke different responses. For example, the weak smell activates fewer ORNs, less strongly.

The best study so far to investigate how odors are encoded at different concentrations was done in locusts by Stopfer, Jayaraman and Laurent. I have covered that paper previously on this blog, but I will summarize the findings.

Stopfer and Jayaraman recorded from projection neurons (PNs) in the locust antennal lobe while presenting odors at different concentrations over three orders of magnitude. They calculated the phase PN spikes compared to the LFP, and found that the phase of firing was the same at all concentrations. They then looked at how neurons responded over the course of the odor, and found that neurons responded similarly to different concentrations of an odorant. For example, PN 8 below has the same response to most concentrations of gerianol. There were some some subtleties, however. Some neurons changed the timing of their firing at different concentrations, like PN 3 which fired earlier at lower concentrations of hexanol. And sometimes neurons responded completely differently to neighboring concentrations, like PN 8's response to gerianol at the highest concentration.

From Stopfer et. al., 2003.
To look at this idea on the population level, they created a population vector that contained the response of a neuron over the entire odor presentation period. They then calculated the distance between the population vectors for different odors and concentrations, and performed a clustering analysis. They found that different concentrations of odors clustered together, but that within those clusters, the highest concentration was farthest from the lowest concentration.

From Stopfer et. al., 2003.
In the last figure of the paper, they recorded from Kenyon cells, and found that some Kenyon cells responded to odors at all concentrations, while others responded specifically to a single concentration of odor. In summary, in the antennal lobe, different concentrations of the same odor are encoded by similar neural representations.

Concentration coding in mammals

For one figure of my manuscript, I recorded mitral/tufted (M/T) cells in awake mice while presenting odors at three concentrations (0.1-2%). However, the manuscript is not about concentration and odor identity, so I will present that data here. In total, I recorded from 105 cells from two mice, and presented three odors, for 315 cell-odor pairs.

What I observed is similar to what Stopfer found: many neurons respond to different concentrations of the same odorant in a similar fashion. However, unlike Stopfer, M/T cells' firing was generally weaker at lower concentrations of an odorant.

An M/T cell's response to 3-hexanone over the range of concentrations. This cell fired phasically in response to the odor, but at lower magnitude for lower concentrations.
Other cell's responses were not so neat. Some cells actually responded more strongly at lower concentrations, like the below cell that received strongers inhibition at lower concentrations. Some cells also became more excited.

This M/T cell is inhibited by the odor more strongly at lower concentrations of odorant. (Also note that post-inhibitory excitation at the highest concentration reflecting the dynamics of the odor response.)
Last year at SFN, Roman Shusterman from the Rinberg lab presented a poster regarding concentration coding in M/T cells. While I did not see the poster, my understanding is that they found that for responsive cell-odor pairs, as concentration increased, the spikes arrived earlier in the sniff. Given that, I decided to see if I could replicate their finding.

In contrast to the Rinberg lab, most of the neurons that I record do not have millisecond precision, but more like tens of millisecond precision. However, I did find one remarkable cell that responded with a short burst of spikes in response to amyl acetate. And indeed, if you look at the spike timing for this burst, the burst arrives ~10 ms later for lower concentrations of odorant.

Top: Raster plot of spikes in response to presentation of amyl acetate at three concentrations. The blue lines at negative times are the last inspiration before the odor (as a control to show that odor duration in the nostril is not effecting timing). As the concentration goes down, the spikes arrive later.
Bottom: Plot of phase of response at high (x-axis) and medium (y-axis) concentrations (odors are amyl acetate, butanol, and 3-hexanone). The medium responses arrive later in the sniff (Watson Willis, p<0 .01=".01" td="td">
To look at this for all cells, I calculated the circular mean (i.e. phase) of firing for responsive cell-odor pairs during the first breath. As reported by Shusterman and Rinberg, I found that the lower concentrations elicited firing later in the breathing cycle (see above). Hopefully their complete story will be published soon, as I'm sure they have a more rigorous analysis.

In summary, neurons in the mouse olfactory bulb respond to different concentrations of the same odorant with similar spike trains. Some (but not all) neurons fire earlier in the sniff for higher concentrations. The similarity in responses provides a neural basis for encoding "odor identity," while the slight changes in magnitude and timing of the responses may allow the mouse to discriminate concentration.

Monday, August 27, 2012

The paper currency of science

OR: Mike reads chapters 4&5 of Wealth of Nations

This is the 3rd in a series of posts wherein I attempt to apply economics principles to neuroscience. Econoneuroscience if you will. Previous posts covered transaction costs and specialization.

While currency (or money) seems simple and obvious - you trade goods for money, and then vice versa - it has great nuance. In the age of physical currency, money ranged from gigantic stones rings to cigarettes to gold. In our digital world, most money is not physical, but simply bits representing our bank account.

Since science is a (haphazardly planned) market, I wondered what our currency is, and perhaps if we could do better. Luckily for me, Adam Smith covered the basics of currency just after considering specialization, which I summarize here.

Adam Smith on currency

In the first three chapters of Wealth of Nations, Smith explored how labour specialization increases productivity, and in chapters four and five, he considered the implications of specialized labor on the market. Specialized labourers produce a "superfluity" of a single good, and must trade it for other goods. For some professions, like bakers, it is easy to trade the surplus: just trade bread for a nail. For others, it is awkward. How does a shepherd get a beer, when all he has are oxen?

Rather than barter we use money. Metals are especially useful for money because they are non-perishable, unlike bread or cows; and metals are easily divisible, so you can trade for the precise worth of goods. (Smith further describes how metals went via bars to become marked coins, and how those coins got debased, but that is irrelevant for us.)

Once a society starts using money, it next needs to find the price of each commodity. Smith argues that the economically "real" value of any commodity is how much labor it took to produce. Thus, if you buy a commodity with money, you are indirectly buying the producers' labor. Now, if you're measuring commodities' worth in labor, how do you then value the labor? Sometimes an hour's work is strenuous, and other times it is easy; and people invest in education and training to increase their productivity. The answer is money: if a commodity requires arduous work, or skill to produce, it will cost more. Thus money allows us to value and exchange labour, the real commodity. (Here, Smith goes on a long exploration of the real and nominal values of labor/money/commodities, which I briefly discuss at the end of this post.)

Putting these ideas together, you can create the classic definition of money: a store of value, medium of exchange, and unit of account. So what fits these properties for science?

Identifying the scientific currency

For simplicity, I would argue that there are two types of agents in the scientific market: labs, and funding agencies. Labs specialize in producing various forms of data, and seek to trade data for funding; the funding agencies "produce" funding, and seek to purchase the best data.

"Now, wait a second Mike, isn't money the currency of science?" This has obvious merit. Labs that produce more data generally get more funding, broadly fulfilling the unit of account aspect. But thinking of funding as a medium of exchange is strange, since funding agencies "produce" funding, rather than exchange something for funding. Indeed, most labs I'm aware of don't trade funding to other labs in exchange for data, which you would expect if funding were a medium of exchange. And funding is a terrible store of value since it runs out in 3-5 years, and labs are forced to spend their entire budgets while they can. While funding is an obvious currency, it does not fit well.

Instead, I would argue that in practice, papers are the currency of science. First, papers are a unit of account. From a lab's perspective, high-profile papers theoretically contain more, higher value, and labor intensive data than low-profile papers; and from funding agencies' perspective, more funding is given to labs with more and better papers.

This also emphasizes the second aspect of currency, namely that it acts as a medium of exchange. Labs trade data for papers, then trade papers for funding. Labs also sometimes collaborate together to produce data for papers. Funding agencies can't directly buy data, due to the circuitous route data production often takes (if only buying desired data was possible!). Instead, they must buy data after the fact, by giving funding to labs that produce papers.

Finally, papers act as a store of value. If I publish a paper in 2012, I will be able to "exchange" that paper for funding or positions, years down the line.

It may be counterintuitive to think of scientific papers as a currency, but they have all the requisite characteristics. There are, of course, many problems with this currency.


Smith noted that metals were commonly used as currency, since they are non-perishable, and easily divisible. In contrast, papers are neither. While a paper published in 2012 retains its value for a few years, that value constantly decreases; a paper published ten years ago will get you little funding or positions today. Indeed, this causes people to constantly trade papers for funding to generate more data; one might even call this inflation. I'm not sure any scientific currency can solve this problem since ten-year-old data is almost always less valuable than new data; the ten year old experiments have already been done (and hopefully replicated).

Papers are indivisible as well; in other words, they work as a poor unit of account. From the top-down perspective, it is difficult to compare the value of papers from different journals. Is a Nature paper worth the same funding as two Neuron papers, and four Journal of Neuroscience papers? Or perhaps we should rank the journals by impact factor, and the papers by citations? Whatever metric ones comes up with will be flawed.

From the bottom-up perspective, it is hard to identify how much a paper's constituent parts are worth. Smith claimed the value of money was how much labor it can command. How much data or labour goes into a paper? Nature papers have 4-6 main figures, but can have over a dozen supplemental figures. In contrast Neuron papers are 6-8 figures long, but have 5-8 supplemental figures. Which required more data? How does one compare different fields? Is a Western blot worth a two-photon image? And if someone uses better technology to get their data, should their paper include more figures or less? These are difficult questions, only made more so by filtering through the oxen of papers.

A new currency?

Biotech companies are lucky, in that they can use actual money as their currency: they produce data which is used to make products that get sold. What are we in academia to do?

Fundamentally, the problem with using papers as a currency is that they're bad units of account: they're too big, and only vaguely tied to value. It's as if we were trading oxen by only knowing their weight, and ignoring their parentage and health.

The size issue is relatively easy to solve: limit papers to just a few figures. Some people denigrate the idea of "salami science," but it's a much more precise accounting. The last paper I reviewed was published in Nature, and had six main figures, and fifteen supplemental figures. In comparison, another Nature paper last year had three main figures, and four supplemental (and much smaller figures to boot; note that both are fine papers, and am simply commenting on size). Wouldn't a more fair accounting system have split the first paper in three? They could even be published in different journals. It would also de-emphasize the pernicious idea of "storytelling," and simply let people publish nuggets of data that may not fit into a grand arc.

The issue of trying to assign value to data is a harder nut to crack. We could try to follow Smith, and measure the man-months taken to produce data. To account for time-saving innovations, we could assign a multiplier to innovative techniques. Yet, how would we account for effort, skill, or the simple dead time in the middle of running a Western? It would be easier to value data post-hoc, rather than summing the labour inputs.

Ultimately, I think the best appraisal of value is the one proposed many times before: let citations and the community weigh the value of data, rather than a few, arbitrarily chosen reviewers. Community rating may be subjective and have its biases - favouring established labs, or flashy results - but science is imprecise enough that I can't think of a better metric.

My core conclusion from thinking about scientific currency - that we need to ditch peer-reviewed papers, and replace them with smaller, post-publication-evaluated data bites (in some form) - is not new. Perhaps this idea is my panacea. Yet, the route was virgin. By looking at science as an exchange between labs producing data, and funding agencies providing money, you can see the fundamental question is how to value data. Regardless of other complaints about publishing - its delays, and arbitrariness - trying to connect data to funding via papers is like trying to run an economy by trading oxen for beer.

(Looking over my notes, Smith has some other interesting nuggets that did not fit into the main narrative of this post. He discriminates between value in use (what a good can do; water has this) vs value in exchange (what you can trade for; e.g. gold is expensive). In science, anatomical studies are often useful, but don't yield high-profile papers. In contrast, many flashy papers get published in Nature and Science, but are often simply wrong.

Smith also distinguishes between the real price (in units of labor) and nominal price (in money) of commodities. These often change with supply and demand, or due to technological innovation. For example, electron microscopy has probably had a stable real and nominal value over the last 20-30 years, and both the real and nominal value of Western blots has cratered due to improved technology. In contrast, the nominal value of imaging has gone up as fluorophores improved, even as the labor necessary to produce images has gone down. This further emphasizes the difficulty in trying to value papers by their inputs.)

Monday, August 13, 2012

Walk Along the Paper Trail: 'Attaboy! Atasoy

On a basic level, feeding (viz. eating) is regulated by opponent pathways: for example the hormone leptin is anorexigenic (prevents feeding), whereas cannabinoids are orexigenic (causes feeding); in the lateral hypothalamus, AGRP-expressing neurons are orexigenic, while POMC neurons are anorexigenic. Many of the players in feeding regulation are known, and the current task in the field is tying them together into a coherent whole. Last month, the Sternson lab published a tour de force paper which takes a step forward in this direction, which I will cover here.

Opponent pathways in the arcuate nucleus

Two groups of neurons in the arcuate nucleus of the hypothalamus play essential, and opposing roles in regulating food intake. POMC-expressing neurons activate anorexigenic pathways, and are stimulated by leptin. The anorexigenic nature of these neurons can be seen from selective ablation of POMC neurons via diptheria toxin, which causes an increase in body weight and food intake. Furthermore, stimulation of POMC-ChR2 neurons causes a decrease in feeding. Multiple lines of evidence connect POMC neurons to leptin: bath application of leptin depolarizes POMC neurons; and selective leptin-receptor knockouts in POMC neurons causes increases in weight.

Selective ablation of neurons causes weight changes. c. Ablation of POMC-expressing neurons via diptheria-toxin causes an increase in weight. a. Ablation of AgRP neurons causes starvation.
From Gropp et al, 2005.
In contrast to POMC neurons, AgRP neurons are orexigenic. Slice work has shown that AgRP neurons inhibit POMC neurons via GABA release. AgRP neurons' orexigenic nature has been shown by ablation of AgRP neurons via diptheria toxin, which causes anorexia, and by stimulation of AgRP-ChR2 neurons, which causes obesity.

Channelrhodopsin stimulation of arcuate neurons changes food intake. a. Stimulation of POMC-ChR2 neurons decreases food intake. e. In contrast, stimulation of AGRP-ChR2 neurons causes an increase in food intake.
From Aponte et al, 2011.
POMC and AgRP neurons project to a few downstream nuclei, but of interest for this paper are AgRP neurons' projections to the paraventricular hypothalamus (PVH), and the parabrachial nucleus (PBN). I don't know much about PVH, but the PBN is the 2nd relay in the taste circuit. A series of papers from the Palmiter lab have implicated that PBN projection is important: in AgRP diptheria toxin mice, you can prevent starvation by implanting a cannula in the PBN that releases GABA agonists.

There are many more players in food intake, including NPY, melanocortin, ghrelin, cannabinoids, dopamine, insulin, and many more. If you'd like to know more, I recommend this solid (but aging) review from Morton. All you need to know for this paper is that AgRP and POMC neurons perform opposing functions in the taste circuit.

Interplay between AgRP and POMC

The main thrust of this paper is trying to understand how AgRP neurons can regulate feeding through their projections using a variety of transgenic, viral, and optogenetic techniques. This paper has 6 main figures, and 15(!?) supplemental figures, so I will only be highlighting the main points.

First, they investigated how AgRP and POMC neurons interact, advancing previous work by using optogenetics.  They cut slices in AgRP-ChR2 and POMC-ChR2 mice, and patched these cells to see how they were connected. As reported previously, they found that AgRP neurons have GABAergic projections onto POMC neurons. However, there was no reciprocal POMC->AgRP connection, nor any AgRP->AgRP or POMC->POMC connections.

Since AgRP can inhibit POMC neurons, they wondered whether the silencing of POMC neurons alone is able to influence feeding. To silence POMC neurons, they used POMC-hM4D mice. If you are unfamiliar with hM4D, it is an artificially developed GPCR that is activated by a molecule called CNO, and reversibly silences neurons. When they gave the POMC-hM4D mice CNO, the mice did not gain weight like diptheria toxin mice (or at least not statistically significantly in 8 mice). Stastitical significance aside, it appeared that inhibition of POMC alone is not able to drive feeding, and thus AgRP probably works primarily through other pathways.

As a final step to investigate the interplay between AgRP and POMC, they used double transgenic, AGRP-ChR2/POMC-ChR2 mice, and stimulated both groups of neurons simultaneously. These mice increased their food intake, showing: 1. that AgRP activity can dominate POMC activity; and 2. that POMC inhibition is not necessary for increased food intake. From this initial set of experiments, they conclude that AgRP neurons do not primarily work via inhibiting POMC neurons.

Stimulation of both AGRP and POMC neurons leads to an increase in feeding. left. Diagram of activated neurons. i. Pellet intake during light stimulation. j. Food intake increases during stimulation of both AgRP and POMC neurons.
From Atasoy et al, 2012.
Investigating other downstream nuclei

To look at AgRP neurons' effects on PVH and PBN, they once again used AGRP-ChR2 mice, but instead of implanting the light fibre over the hypothalamus, they put the fibre over the axons in the PVH and PBN. When they stimulated AgRP axons in PVH, they saw an increase in food intake, showing that AgRP->PVH activity is sufficient. However, when they stimulated the AgRP fibres in the PBN, they did not see an increase in food intake. Thus, of AgRP neurons' three possible targets, they hypothesized that their PVH projection is most important for food intake.

Stimulation of AgRP fibres in PVH is sufficient to increase food intake. top. Experimental setup and food intake for PVH stimulation. bottom. Experimental setup and food intake for PBN stimulation.
From Atasoy et al 2012.
Focus on PVH

Having identified PVH as important, they homed in on it. First, they explored the AgRP-PVH connection in slices, and found that there is indeed strong inhibitory input from AgRP to PVH. Then, to see whether PVH inhibition is sufficient to induce feeding, they expressed hM4D throughout PVH by using the SIM1 promoter (SIM1-hM4d). Upon administration of CNO, these mice gained weight, showing that inhibition of PVH is sufficient. Since no one would believe a single silencing paradigm, they repeated the experiment using PSAM-GlyR, and saw the same effect. To show that PVH inhibition is necessary, they created AGRP-ChR2/SIM1-ChR2 mice, and stimulated both populations simultaneously, and this was not able to increase food intake. Thus, PVH activation can prevent AgRP-neuron induced feeding, and PVH inhibition is necessary for AgRP-neuron induced feeding.

PVH inhibition is sufficient and necessary for increased food intake. b/c. In SIM1-hM4D mice, CNO administration causes increased feeding. e. Diagram of double stimulation experiment in AGRP-ChR2/SIM1-ChR2 mice. f. Double stimulation does not case an increase in feeding.
From Atasoy et. al., 2012.
The PVH contains multiple types of neurons, and of these, they decided to focus on the oxytocin (OXT) expressing neurons. They again performed the double stimulation protocol, this time in OXT-ChR2/AGRP-ChR2 mice, and again found that OXT neuron stimulation could prevent AgRP-neuron induced feeding.

In the final set of experiments, they investigated whether AgRP neurons release neuropeptide Y (NPY) and GABA in the PVH. To do this, they implanted cannulas with pharmacological antagonists for each of these neurotransmitters in the PVH of AgRP-ChR2 mice. Blocking either neurotransmitter decreased the AgRP-ChR2 induced feeding, showing that both neurotransmitters are functional at the AgRP->PVH synpase.

Publishing thoughts

Phew! I told you that was a tour de force. By my count, they used eight(!) transgenic mouse lines, and five different viruses. They nonchalantly mentioned results that might be a starting point for a whole paper in a single sentence, "we have found that food deprivation increases inhibitory synaptic drive onto PVH neurons (Supplementary Fig. 12)."

To be honest, the sheer magnitude of this paper kinda pissed me off, since the results could have come out sooner if the paper was split in two (yet more evidence of Nature's supplemental figure problem). This paper was received by Nature last September, accepted in May, and published in July; it took ten months for this to become public. Everyone's ok with this?

Scientific thoughts

Given the sheer number of experiments in the paper, I was somewhat disappointed by the two paragraph discussion. To be fair, this is probably due to the six page limit (which would explain the above mention of Sup. Fig. 12, and yet another reason to dislike journals). For example, as I mentioned in the background, there is evidence that AgRP neuron GABAergic signaling to the PBN is necessary for normal feeding. However, the PBN gets a single sentence in the discussion, "Finally, AGRP neuron projections targeting the parabrachial nucleus (PBN) in the hindbrain do not directly activate feeding, but instead they restrain visceral malaise that results from AGRP neuron ablation." Those Palmiter papers also investigated the PVH, and found that it was not important in their paradigm, so I would really like to have seen a more thorough exploration of the differences between the papers.

The most intriguing single experiment, to me, is the dual activation of both AgRP and POMC neuron populations, which implies that the orexigenic pathway is able to dominate the anorexigenic. If I may speculate, when humans talk about satiety, we range from hungry to sated to full. Hunger is a strong feeling, motivated by blood sugar levels (or something), while fullness seems more "visceral," governed by stomach distension. However, satiety is a rather subtle feeling, since it is the default (at least in the developed world). Perhaps the argument of origenic vs anorexigenic pathways is entirely wrong, and the actual opposition is between orexigenic and non-genic pathways. If provoked, we can feed while we're sated, but if we're full, stuffing more food in our face is nauseating. In any case, I look forward to their undoubtedly ongoing experiments looking at POMC's projections, and to see how those projections overlap (or not) with AgRP.

While I know vanishingly little about oxytocin (I leave that to cognitive scientists), in the discussion they note that oxytocin disorders in humans can lead to "instiable hunger." What I find strange is that the body would transduce a straightforward satiety signal (leptin/cannabinoids) into another hormonal signal, oxytocin; unless, of course, oxytocin here is simply a neurotransmitter, and not an endocrine signal. Of interest to me is that oxytocin is expressed by the glial-like taste receptors on the tongue. While these glial-like cells do not have taste receptors themselves (the receptors are in the aptly named receptor cells), it is possible that oxytocin could indirectly modulate taste itself, similar to how leptin and cannabinoids can directly modulate sweetness.


Atasoy D, Betley JN, Su HH, & Sternson SM (2012). Deconstruction of a neural circuit for hunger. Nature, 488 (7410), 172-7 PMID: 22801496

Monday, August 6, 2012

Is neuroscience a meritocracy?

Meritocratic failure

In a recent article, "Why Elites Fail," Christopher Hayes argues that meritocracies often fail. For example, the Hunter College High School, a prestigious public school in NYC, accepts students solely on the basis of an entrance exam. For decades, the meritocratic admissions process meant that the school had a diverse student body: in 1995, 12% of the school was black, and 6% hispanic. Today, however, when rich parents hire private tutors for their kids, the student body has become less diverse, with 3% black, and 1% hispanic.

Hayes identifies two keys to meritocracy. First, there must be interindividual differences in ability, skill, or what-have-you. Second, people must be rewarded for their performance: high performers get promoted, while low performers are "punished." If you looked at how families perform between generations, variance in individual ability and accountability would cause inter-generational mobility: if a parent is exceptional, and their child average, the family would change positions. In theory, the larger the variance in interindividual differences is, the larger the mobility should be (I am not sure how to state this formally and correctly, but that is Hayes argument).

However, this is not what usually happens. After one round of meritocracy, the winners get to invest in the next round, so their children have advantages. Sometimes the winners, often in authority, get to choose the winners (or losers) of the next round. The meritocracy breaks down, and become an oligarchy. This same basic story has played out in many different fields in the last thirty years, including college admissions ("legacy admissions"), or the decrease in inter-generational income mobility in the US since 1970.

After reading the article, I wondered, has the meritocracy has failed in neuroscience as well?

How to test scientific meritocracy

The scientific meritocracy can fail in many ways. Funding can go to prestigious labs, regardless of how efficient they are. Nobel laureates can publish shitty papers in high profile journals. Prizes can go to PIs at "elite" institutions, to reflect how portentous the prize is. (I use "elite" as a shorthand for the top 5-10 neuroscience programs, based on my rankings, and my perception. Exploring what constitutes an elite institution is for another post.)

Since I pretend to be a scientist, I wanted to measure how meritocratic "science" is. One possibility would be to measure whether labs from elite institutions get preferential treatment by journals, but that would require somehow objectively measuring paper quality, and controlling for funding. Instead, I settled on something simpler, and hopefully more objective: looking at the credentials of PIs at elite institutions.

In a meritocracy, you would expect that as you looked farther into the past, the winners would have increasingly diverse backgrounds. Or, looking forward, you would expect some percentage of people who start at the bottom of the hierarchy, but were talented, could work their way up. In terms of PIs at elite institutions, my expectation is that they would almost exclusively have done post-docs at other elite institutions. In contrast, if you looked at where they went to undergrad, I would expect a more diverse set of schools. To see whether this was true, I looked at 30+ PIs at elite institutions.

The test

The baseball writer Rob Neyer has a gimmick where he presents stats for two anonymous players, "Player A", and "Player B." Sometimes the stats would be similar, and the reader would be shocked to find that Player A was an All-Star, while Player B was a "scrub." Sometimes the stats would be dissimilar, but Players A and B would actually be the same player, playing under different conditions. The point of the gimmick was to look at the world more objectively, without the halo effect of their names.

So in that vein, I present two cohorts of researchers, and their associated institutions:

Cohort A Cohort B
Santa Barbara
Cal State Chico
Williams College
Bryn Mawr

With all due respect to the universities represented in Cohort A, most people would agree that the schools in Cohort B produce more research. So who are these two cohorts of researchers, and how are they affiliated with the institutions? Both cohorts are professors at elite neuroscience departments. But Cohort A got their bachelors before 1990 while Cohort B got theirs after 1990:

Fogies Undergrad Whippersnappers Undergrad
Barres, Ben
Newsome, William
Moore, Tirin
Raymond, Jennifer
Shatz, Carla
Nicoll, Roger
Huganir, Richard
Bear, Mark
Julius, David
Malenka, Rob
Tsien, Richard
Katz, LC
Callaway, Ed
Cline, Hollis
Santa Barbara
Cal State Chico
Williams College
Bryn Mawr
Datta, Bob
Wilson, Rachel
Ehlers, Mike
Scott, Kristin
Harvey, Christopher
Dolmetsch, Ricardo
Heiman, Miriam
Huberman, Andrew
Potter, Chris
Shuler, Marshall
Tye, Kay
Goosens, Kim
Sabatini, Bernardo

(Methods: To select fogie professors, I included professors I recognized by name, or who are HHMI investigators. For the whippersnappers, I used the SFN Young Investigators award listing, and scanned the websites of departments for assistant professors. People educated outside the US were excluded. For each professor, I noted their schools for BS, PhD, and post-doc. For some professors, I could not ascertain their undergrad institution, and excluded them. This is by no means exhaustive, but I only spent a few hours doing this. Full spreadsheet.)

I have two general conclusions from this. First, if you want to be a professor at an elite institution today, you need to have gone to an elite undergrad. The "worst" school represented, UVA, is ranked #25 (for whatever rankings are worth). There are no Wisconsin-Madisons on the list, let alone places like Ohio State. Three steps removed from your PI position, where you did your undergrad is a determining factor for whether you can become a PI. As you move the credential window forward to grad school and post-docs, the credential threshold gets even higher.

Second, and more weakly, I think this shows that science has become less meritocratic over time. At first blush, I thought the fogies' schools were just generally worse. A quick Google, however, revealed that Williams, Vassar, and Bryn Mawr are well regarded liberal arts schools. So while it's not fair to conclude that the older cohort went to worse schools, I think it is fair to say they went to a more diverse set of schools, ones that did not necessarily emphasize research.

Questions I ask myself

Isn't this sample size small? Yes, but then again there aren't many professors. If I wanted to spend more time, I should probably quantify rankings of both the PIs' institutions, and their undergrad schools. It would also be helpful to look at non-elite institutions' PIs to see how what is happening there.

Don't elite undergrads reflect high SAT scores/intelligence? And science is a g-loaded occupation, so... I would argue that neuroscience is not as g-loaded as other fields like physics or computer science. Once you reach +1 or +2 SD, other factors like work ethic become important. A quick Google shows that elite institutions have, a ~100 point SAT score gap over other places, ~1SD. However, there are much fewer elite institutions, and they also have fewer students, so by numbers alone, there should be just as many equally smart kids at non-elite institutions as the elite ones, and a much larger group at -1SD.

Ok, if the elite kids aren't smarter, could they have some other trait? I can see this. College admissions are increasingly (and insanely) competitive. Thus, elite colleges may screen for competitiveness. If not competitiveness, it could be some other factor. Elite institutions use extracurriculars as differentiators between applicants, and if you believe this Gladwell piece* the extracurriculars reflect something real.

* What happened to Gladwell? People love to shit on him now, but I thought he did a good job summarizing social science for a wide audience in The Tipping Point and his older magazine articles. But now he's publishing half-baked essays on "slack," and name-checking Tyler Cowen's econ-foodie book?

What about non-Americans? I don't know much about this. My understanding is that the European college system is much more equal in terms of quality, so there is less fighting for spots (with exceptions like the French/Swiss écoles, Oxbridge,  etc.). As for Asia, I think a majority of Chinese PIs in the US come from Tsinghua or Peking University, and the Indians from IITs. But American grad schools may not be equipped to identify good applicants from less famous schools.

Isn't this a lot of words for what amounts to glorified googling? Yeah.

Concluding bloviating

As mentioned at the top, society as a whole is becoming less meritocratic. It would be remarkable for science to resist this trend. I'm not sure what, if anything can be done about it. The PIs at elite institutions are generally smart, motivated people, so from the perspective of funding agencies, why should they care whether the PIs have diverse backgrounds? And the NIH does fund non-elite institutions, just less so, if only to avoid senators asking why Idaho doesn't get any funding.

There is an opportunity for disruption here, in that elite institutions are completely overlooking talented, less credentialed people. Some places, like Washington University, seem to specialize in being less famous, but nearly as productive, and I think it's in large part by finding people the elite institutes can't be bothered with. Of course, they will still lose status contests to elite institutions in publishing and prizes.

On a personal note, I knew coming to Geneva that I would need to do a second post-doc to get a job back in the US. Seeing the credentials of these people made me realize just how important status and political connections are, rather than simple productivity. Hopefully, the status requirement are much lower one step down the ladder. The post-docs I knew at Duke were able to get positions at good schools like UNC, Baylor, and BU. Whether they would have been able to get the positions if they were post-docs at those schools is another question.

Thursday, July 19, 2012

Synaptic Diseases 2012

The University of Geneva, and more specifically the Luscher lab, hosted a conference last week called Synaptic Diseases. Since I never got to go to a small conference when I worked in synaptic neuroscience, I decided to attend. The presenter list was stacked, including Malinow, Malenka, John Isaac, Nicoll, Sabatini, Kauer, Bredt, Tomita, and many more.*

Never invite pharma guys to speak

Pharmaceutical companies have poached many neuroscientists: Marc Tessier-Lavigne, Mike Ehlers, and attending this conference, Michael Hutton, David Bredt, John Isaac, and Bai Lu. And I should preface the next sentence by saying all of these men are ten times the scientist I am.

These guys gave terrible talks.

I understand life is different "in industry." Your data is now proprietary, not public domain. You need to shield the specifics of your work from competitors. And as the head of a pharma division, you may not know the nitty-gritty details of all the experiments. But, with the exception of Michael Hutton, the rest of their talks included zero data (and Hutton's was more of a review than a talk). One guy (specific names withheld) talked about potential drugs for glutamate receptors, which consisted of showing chemical structures of existing drugs, and 3D reconstructions of receptors. Another guy talked about how BDNF might be used as a therapy, and how they want to investigate it from the genetic (microarrays!) to the brain level (EEG!). One of them even looked ashamed, averting his gaze to the ground, and stumbling over words.

To me, the whole point of a talk is to present your preliminary, unpublished data. The audience gets the thrill of seeing something new, and the presenter can get feedback. Since (I assume), these pharma guys aren't allowed to present preliminary data, I would recommend never inviting them to give a talk.

With that said, here are the highlights from two talks.


Classically, people think that the difference between inducing LTP and LTD induction is calcium levels: above resting calcium, moderately high calcium induces LTD; while even higher calcium induces LTP. The calcium source for both LTP and LTD (at the hippocampal Schaffer Collateral) is the NMDAR; and indeed you can block LTD with the NDMAR antagonist APV.

For whatever reason, Malinow's lab tried to induce LTP while blocking NMDARs with MK-801. MK-801 is a competitive antagonist that binds to NMDARs differently than APV, and blocks NMDAR currents. And when they tried to induce LTD with MK-801, they could. When they tried to induce LTD with a glycine blocker (glycine is an obligatory co-agonist for NMDAR), they could again induce LTD. Their working hypothesis is that glutamate binding to NMDAR is necessary for LTD, but not ion flow through the receptor.

What about calcium? One reason people think calcium is essential for LTD is that if you include high affinity calcium buffers (BAPTA) in your patch pipette, you can block LTD. However, calcium buffers have the side effect of lowering resting calcium. To see whether changes in resting calcium were influencing their results, Malinow's lab compensated for the buffering by including extra calcium in the patch pipette (15mM BAPTA + 1.3mM CaCl2, which they found via calibration of AMPAR currents), and were able to induce LTD. Their interpretation is that some resting calcium is necessary for LTD, but that calcium flow through NMDAR is not.

Since this is a Synaptic DISEASE conference, they had a disease tidbit. Alzheimer's Disease is in part caused by amyloid beta, which comes from amyloid precursor protein (APP). APP can induce LTD when applied in slices, and this LTD can be "rescued" by APV. However, they found that this LTD cannot be rescued by MK-801 or their glycine blocker. In the elevator after the talk, one person claimed that Malinow has been giving versions of this talk for two years.


AMPAR function is enhanced by auxiliary subunits, including TARPs, and cornichons (CNIH). In drosophila, it's know that CNIH is expressed mainly in the Golgi, and probably plays a role in ER exit.

To investigate CNIH function in mice, Nicoll's lab employed an approach similar to their recent Lu paper: they floxed CNIH, and then knocked it (the fuck) out via sparse viral infection of Cre. CNIH-2 KO neurons lost 50% of their surface AMPAR expression, both synaptic and extra-synaptic; CNIH-3 KOs were unaffected; and CNIH-2/3 2KOs lost 80% of surface AMPAR.

To investigate which AMPAR the CNIH were interacting with, they applied CNIH shRNA in GluA1-KO mice, and found that there was no effect of the shRNA. However, in GluA2 KO mice, CNIH did reduce surface AMPAR. Furthermore, Western blot pulldowns showed that CNIH binds to GluA1, but not GluA2. Thus it appears the CNIH selectively bind to GluA1.

To check the CNIH-GluA1 interaction a third way, they expressed GluA1/2, CNIH, and TARPs (y-8) in HEK cells, and measured the desensitization kinetics. GluA1/2 heteromers have longer taus than GluA2/3 (4ms vs 2ms). And here my notes fail me, but the gist of it is that for GluA1, both TARPs and CNIH can bind without interfering with each other. However, for GluA2, TARP binding is dominant, and prevents CNIH binding. Finally, they did a glycosylation assay that shows CNIH-KO mice have immature AMPAR.

Their working hypothesis is that CNIH is involved in ER exit, as it is in the fly, and that CNIH binds to the GluA1 of GluA1/2 heteromers. During the question time, both Malenka and Tomita asked pointed questions about possible compensation: for the CNIH-2 shRNA experiments, by the other CNIH; and for the TARP experiments, by other TARPs.

There were a lot of other good talks, but a lot of them weren't about canonical AMPAR/NMDAR/LTP/LTD, and I can't do them justice here. With no more conferences on the horizon, it's time to get back to the paper trail, both mine and others.

* Comparing this conference to ISOT, it feels like the synaptic physiology field is a lot less gender equal than olfaction. In Geneva, there were 6-7 women out of ~35 presenters. In comparison, at ISOT, there were major talks by Buck, Vosshall, Kristin Baldwin, Kristin Scott, and Bargmann, and those are just the people I can name off the top of my head. Not great, but better. Of course, that's nothing compared to the underrepresentation of black people (2/600 at ISOT and 1 at Synaptic Diseases).

Monday, July 9, 2012

Compendium of Analyses, Part III: Advanced Single Cell

I wrote the first two compendiums of analyses last year covering single cell and population analysis. Since then, I've been keeping track of the more esoteric analyses I've encountered, and also found a short perspective from Mitra that covers many spike train analyses. Today, I'll cover three analysis techniques: sparseness, ROC analysis, and entropy.


The simplest, first. I originally found the sparseness calculation in Poo and Isaacson, 2009, where they calculated the sparseness of odor responses in piriform cortex. Sparseness simply measures how much of a stimulus space a neuron responds to, between 0 and 1 (1 being highly sparse). The equation is quite simple:

Sparseness equation.Where ri is the firing rate of the neuron to stimuli i, from the set of stimuli n. From Rolls and Tovee, 1995.
If a neuron responds to single stimuli in a set, and does not fire for the rest, the numerator and denominator are equal, and the sparseness is 1. If, however, a neuron fires at 1Hz to each of 10 stimuli, the sum of the numerator would be 1/100 * 10 = 0.1; and the denominator would be 1/10 * 10 = 1; and the sparseness would be 0.1. Poo and Isaacson found that the sparseness of piriform cortex neurons was 0.88, indicating most cells there responded selectively to a single odor. You can also look at population sparseness, using the response of a cell to a specific stimuli, in a population of cells n.

A more interesting example comes from vision, from Vinje and Gallant. They recorded from visual cortex while presenting movies; the movies varied in size, with the smallest only stimulating the classical receptive field (CRF), while the largest movies covered an area 4x the CRF. For the small movies, the firing rate was moderate, and increased following some frames (below, left). For the large area movies, however, the firing rate was near zero except for specific frames.

Stimulation of larger areas increases sparseness of V1 neurons. Left. Recordings from macaque V1 during playback of movie in the CRF or an area 4xCRF. The firing rate was higher during the CRF stimulation, and more sparse for the 4xCRF stimulation. Right. Sparseness calculation for populations of neurons for stimulation of different size CRFs. As the stimulus area increased, the sparseness also increased.
From Vinje and Gallant, 2000.
To look at this for the population, they calculated the sparseness of the response, where each stimuli was an individual frame. For stimulation of only the CRF, the average sparseness was 41%; stimulation of 4xCRF yielded a sparseness of 62%.

Sparseness seems like a quick and dirty measure, but of limited application to olfaction or taste. First, chemosensory stimulus spaces are generally small enough that you can simply quantify the percent of stimuli a neuron responds to. Second, olfactory responses are highly phasic (or tonic-phasic as one guy at my poster at ISOT insisted), which makes measuring firing rate less useful. To adapt it to olfaction, you would probably need to substitute a different metric for responsiveness, such as "difference in spikes from control breath."

Receiver Operator Characteristic (ROC) analysis:

ROC analysis is used to tell to what degree, and under what conditions, two responses are discriminable. I first came across this in Cury and Uchida, and they have a nice supplemental figure that shows how it's useful in OB coding.

They recorded from the OB of rats while presenting various odors, and wanted to know: 1. whether a neuron fired differently during odor presentation than pre-odor breaths; 2. when the firing was most different; and 3. what time window allowed the best discrimination. To do this, they compared the firing rates between control an odor breaths during defined windows during the breath (red and blue areas in panels A/B below). For each trial, they counted the number of spikes in the epoch (panel C), and then plotted the distributions of spike counts for each trial (panel E).

Copied from Cury and Uchida:
"(A) PETH of an example M/T cell, aligned by the first odor inhalation onset. Black, odor; gray, blank. The red (1) and blue (2) shaded areas indicate two example analysisepochs. This neuron and these epochs are used in all subsequent panels.
(C) Raster plot of spike trains over multiple trials, aligned by the first odor inhalation onset. Odor trials (above) are separated from blank trials (below) by the horizontal black line.
(D) Response reliability (area under the ROC curve, auROC), calculated for varying epochs (bin size: 5 ms to 160 ms.; bin center: t = 0 to t = 160 ms), as in Figure 2D. auROC values are indicated using the color scale at right, with red and blue signifying increased and decreased spike counts, respectively. The black circles indicate the three example windows plotted in (B). Selection of the optimal epoch was restricted to occur within the 0 to 160 ms response window, the bounds of which are indicated by the diagonal dotted lines.
(E) Distribution of single trial spike counts for both odor(black) and blank (gray) for the same three example epochs.
(F) The corresponding ROC curves for the same three example epochs, comparing the hit rate to the false alarm rate as a discrimination threshold is slid across the distributions. The resulting auROC value is listed to the right."
Then, they asked, if you had to classify a trial as "blank" or "odor" based on the number of spikes in a trial, how successful would various thresholds be? For example, in the top part of panel E, a threshold of 0.5 spikes would correctly classify most odor trials (a "hit"), while mis-classifying a few blank trials as odor ("false alarms"). By contrast, for the bottom part of panel E, no threshold could discriminate blank from odor.

For each threshold, you get a proportion of hits vs false alarms, and can plot this as the ROC curve (panel F). If two populations are easy to discriminate, you will get curves like those shown in blue or red; if the populations are hard to discriminate, you get a curve like that shown in black. You can collapse the ROC curve into a single measure of discriminability by integrating the area under the curve (auROC); values near 0 or 1 show strong discrimination for that epoch.

So that's how you get the auROC for a specific epoch. You can then repeat the procedure for different epoch lengths and onsets (panel D). This panel shows that there are two good epoch to discriminate odor from blank: ~60ms, and ~140ms, and that the optimal epoch size is 60-80ms. In general, they found that excitatory responses could be found throughout the sniff, with epoch sizes of ~40-60ms; in contrast, inhibitory responses tended to be between 80-120ms, and have epoch sizes of ~60-80ms.

ROC analysis is kinda funky, so I recommend playing with this demonstration to get a better feel for ROC curves.

Entropy/ information / bits!

I must admit, I don't completely grok entropy. I (think I) understand the equations, but I don't understand what it's useful for. Calculating how much information is in a neuron's firing without decoding the stimulus seems like calculating the optimal bit rate for an mp3 without listening to the song.

In any case, the entropy of a firing rate can intuitively be understood as the unpredictability of the firing rate, and by inference how much information is in the firing rate. The relationship between unpredictability and information is easy to understand: a neuron that fires at exactly 10Hz in response to all stimuli is conveying no information; in contrast, a neuron that fires at 10Hz to one stimuli, but not to others, contains some information; more subtly, a neuron that fires on average at 1Hz, but with bursts and silent periods, depending on the stimulus, also contains information with each spike.

So how do you calculate the entropy of a spike train? (The best review of this I found was from Bhumbra and Dyball). First, you need to bin the spike train into a histogram where each bin can contain only one spike. This histogram has three features: the number of bins, the bin size, and the number of spikes in the histogram. The likelihood of a given spike train is defined by how often that pattern occurs out of all possible spike trains. If a neuron uses all possible spike trains, the entropy of all spike trains is simply log2(# of spike trains possible). Since calculating the potential number of spike trains involves factorials, and is unwieldy for large numbers of bins, this equation can be approximated as (skipping all the algebra here) log2(e/(M*dt)), where M is the mean firing rate, and dt is the bin size.

Of course, neurons don't use all possible spike trains possible, but only a subset of them. In that case, you simply multiply the probability of a spike train times that train's information:

The entropy of a spike train is the probability of a spike train i, times the log of that probability. From Wikipedia entry for entropy.
"Ok, fine, I can look up the details of how entropy is calculated later. What's it good for?" I searched a bit for papers using entropy, and they seem to fall into three categories: comp neuro papers about how to calculate it; papers that use it for a panel in one figure; and vision papers.

To get a good measurement of entropy, you need to record from a neuron for a long time to get a large set of possible spike trains, preferably including many stimuli and trial repeats. This makes entropy ill suited for olfaction, where the unit of measure is one breath (~320ms), and odor presentation can take ten seconds for odor loading and clearance. Vision, however, does not have these problems: you can play a movie to a neuron, yielding 30Hz of stimuli, and can repeat this hundreds of times.

One of the earliest, most cited entropy papers in vision is Nirenberg et al, 2001. They recorded from many neurons from mouse retina while presenting movies 300 times. Some sets of cells had correlated firing, which they calculated as the correlated spikes that would appear above chance, the excess correlated fraction (ECF). To see whether these correlated spikes were informative, they calculated the entropy of the neuron pairs' spike train both including the correlated spikes, and excluding them (or rather, treating them independently). They found that even for neuron pairs with high ECF, the correlated spikes only contained <10% of the total information.

Retinal ganglia act independently. They calculated the information in a neuron pairs' spike train both including and excluding correlated spikes (y-axis), and compared this to neuron pairs' correlation (ECF). While higher ECF pairs' correlated firing did include some information, most information was independent of correlated firing.
From Nirenberg et al, 2001.