After an interesting and wide-ranging discussion, we identified both general
and specific suggestions that may be relevant to this question. These points
of interest may be summarized as (a) the importance of clinical, as well experimental,
work in this process; (b) potential pitfalls to avoid when talking to experimentalists;
(c) the possible opportunity presented by data sharing requirements within the
funding process; (d) the potential of using ‘grand challenges’ to
galvanize research for both communities; (e) some success-stories of bringing
involved computational or analytical approaches to an experimental subject;
finally (f) preliminary suggestions for a targeted workshop/retreat called lightheartedly
‘the carbon / silicon challenge’.
Almost the very first point raised was that clinicians must be included as well
as experimentalists in our discussion to broaden the utility and impact of modeling
work. The natural challenge presented by this is to identify a clinically relevant
issue for which there is a good experimentally-testable system, and identifying
how Biologically Accurate Modeling (BAM) could be used might be the key to encouraging
experimentalists, clinicians, and their funding agencies to adopt BAM-based
approaches. Naturally, physiological disorders (such as Parkinsonism) are more
amenable to this approach than psychological or psychiatric disorders (such
as schizophrenia), since cellular-level approaches use BAM standard tools (such
as GENESIS and NEURON). High-level approaches may still provide experimentally
testable hypotheses that are important for the study of complex psychiatric
disorders and finding ways to bring these approaches into the modeling framework
would be extremely valuable.Models lacking a strong tie to the underlying physiology
are of potentially less interest to funding agencies such as NIH, and may be
more appropriately directed to other Federal funding agencies.
Communication between experimentalists and modeling specialists is a key question.
In order to avoid prejudicing workers from other fields against the use of models,
we felt that the following recommendations be mentioned. (a) making sure the
‘grandmother test’ is fulfilled (i.e., to be able to explain their
model simply enough for their grandmother to understand), (b) describing unambiguously
the essential value-added when modeling is incorporated into empirical studies,
and making sure that our work is not perceived as an esoteric exercise in a
separate field, (c) making sure all participants in a collaboration feel that
they are equal partners in the shared endeavor. For their part, we recognize
that more and more academic programs require computational, informatics-based
components, so that the old story of being able to ‘take biology as a
way of avoiding math’ is dying out.
The latest requirement for data-sharing from NIH may offer modelers access to
experimental data. Under this new rule, researchers submitting a grant of more
than $500K per year to NIH must include a data-sharing plan in their proposal.
In the long term, it is likely that this will expand to include all NIH-funded
work so that researchers will be required to make their data available to other
researchers (after a period of time to permit them to complete their own analyses
and use of their data, e.g., publication or other uses). The time-frame of this
transition is unspecified, but expected to occur in the next few years.
Some computer-science communities use ‘grand challenges’ and competitions
to propel the subject forward. The Text Retrieval and Extraction Competition
(TREC) is one example from the Information Retrieval community. In this situation,
researchers are presented with a corpus of data and asked to perform specific
computation on it which may then be scored. This provides a metric for success
within the subject that provides clear metrics for success. The competition
is well-attended and drives excellence within the discipline forward in a measurable
way. The presentation of suitable ‘grand challenges’ within this
community also galvanizes interest and invites external media coverage. Are
there any such competitions or grand challenges that may be identified within
the BAM community? Could such challenges be started on a shoestring budget?
How would we structure the ‘reward’ system to encourage participation?
The processing of encouraging researchers to include modeling approaches may
be more of an academic exercise appropriate for graduate students, postdocs,
and perpaps even undergraduates. It could begin to involve students at very
early stages of their careers, and pave the way for a paradigm shift away from
solely empirical or modeling careers and into an era of broadly-trained scientists
who are capable of doing both the experimental and theoretical work –
all within a single brain.
We considered some success stories: notably epidemiology and the Statistical
Analysis of Neural Data (SAND) group headed by Dr Rob Kass (the Head of Statistics
at Carnegie-Mellon) and Dr Emery Brown (from Harvard Medical School and the
Harvard/MIT Division of Health Sciences and Technology). His main research interest
lies in the development of encoding and decoding algorithms: mathematical techniques
to decipher how ensembles of neurons represent and transmit information about
pertinent signals from the outside world. This group brings together statisticians
and neurobiologists to find problems of interest for both communities. The final
report of this workshop may be viewed at this URL:
http://www.psc.edu/biomed/training/workshops/2004/sand2/SAND-Conference_Summary_Report_Final.pdf
The SAND group may be able to advise organizers within the BAM community to
organize the ‘crossing the carbon / silicon divide’ workshop. The
strategy behind this idea is to target specific experimental researchers (or
communities) who have data that are directly amenable to modeling methods but
are either not aware of possible modeling approaches or lack the expertise to
use them. One such group could be computational neuroanatomists that have reconstructed
their experimental neurons with tracing packages such as Neurolucida. The workshop
would therefore be based on invitations to specific experimentalists. It could
conceivably be a separate retreat on its own with a keynote speaker with strong
influence in both communities (such as Larry Abbott). It might also be an excellent
feature of future WAM-BAMM meetings where a portion of a day could involve presentations
and tutorials from non-modeling but modeling-relevant speakers.
Finally, it may be productive to publicize the WAM-BAMM event at the SFN meeting,
possibly within some space linked to the NIH/NIMH booths. Another possibility
might be to organize and schedule a satellite symposium at the SFN meeting,
along the lines of the Dynamical Neuroscsience [DN] meeting (or even use the
DN meeting as a springboard for future meetings). Also disseminating information
about this meeting to mailing lists targeted at non non-modelers may be useful.
Having these external structures to the WAM-BAMM meeting itself might be a good
way to also establish cross contacts: a contacts list could be available for
people. Sections on the list might include a “have models, will travel”
section and a “single experimental scientist, modeling curious”
section. Mechanisms for encouraging crossover would be welcome especially to
reinforce communication after the meeting is over.
1. Challenge of Modeling the Whole Small Animal Brain
Microscope technology, both LM and EM, for mapping the morphology of the mouse
brain and other small animal brains has recently taken a significant leap forward.
Is the modeling community ready for modeling the whole small animal brain? What
are the limitations? Can they be transcended in the next five years?
Computer power, though possibly taxing current supercomputer capability, was
not felt to be a significant bottleneck. But other issues, some discussed below,
were. These include:
• Reluctance of modelers to simulate larger networks
• Lattice representations of brain networks (morphology and interconnection
topology)
• Need for NeuroML definition of brain networks
• Need for fiber distributions
• Distribution of axonal delays
• Improved proximity labeling of putative synapses
• Experimental verification of synaptic weights
• Physiological verification of modeling in the intact brain
Each of these topics is discussed briefly below, as having applicability outside
modeling of the entire small mammalian brain.
And finally, there is that nagging question:
• What can we do with the connectivity data once we get a full set?
2. Reluctance of modelers to simulate larger networks
We draw attention to the current lack of interest in simulating larger networks
even though the computing power is available. On the computer science side,
part of this reluctance stems from the inherent difficulty of parallel/distributed
simulation; on the neuroscience side, from the lack of an appropriate theoretical
framework for scaling large scale simulations.
3. Lattice representations of brain networks (morphology and interconnection
topology)
Brain networks, where possible, should be modularized by the explicit introduction
of basic circuits. This modularization imposes a (stochastic) lattice structure
on the brain network, which should subsequent simplify parameter search and
parallelization of the computation.
Shepard’s Synaptic Organization of the Brain, 5th edition, Oxford, 2004,
describes basic circuits and the lattice structure for the 10 better understood
regions of the brain. The cell types for each basic circuit are identified;
generally, cell packing parameters and their lattice are not specified. Few,
if any, wiring diagrams are given for the associated pathways, such as cortico-cortical
connections, cerebellular peduncles, etc.
An empirically-found brain network is at best a “model” in as it
can be constructed only from the superposition of multiple brains and staining
technologies. The methodology to project from an observed brain network (neuron
distributions, their morphology, and interconnection) to the topology of a modeler’s
network (multi-compartmental models, event-driven spike processing) is still
primitive. We need a theory for Levels of Detail (LOD), much as exists in computer
graphics for objects seen up close to the same object viewed as distance.
4. Need for NeuroML definition of brain networks
The definition of the Net component within ML was deemed primitive, if indeed
it exists, and totally inadequate for a whole small animal
5. Need for fiber distributions
Modeler’s complain that they cannot build biologically accurate models
in the absence of empirical data of fiber distributions. Even without detailed
synaptic information, the general fiber density and pathways in the white matter,
for example, linking cortical areas, can be an extremely valuable resource.
Newer techniques, such as tracing osmium tetroxide stained myelinated axons,
have the potential to map fibers in the peduncles connecting to the cerebrum.
6. Distribution of axonal delays
Spike transmission is normally simulated by discrete event-driven networks.
Axon diameters, as measured in LM, are too inaccurate to determine axonal delays.
Axon diameter measurements using SBF-SEM, calibrated by micro-electrode recordings,
may be adequate.
7. Improved proximity labeling of putative synapses
Help is on its way, as rules for declaring putative synapses (as observed in
LM) can be verified in datasets generated by Denk’s Serial Block Face
Scanning Electron Microscope. However, brain networks are dynamical systems,
and synaptic weights can change.
8. Experimental verification of synaptic weights
No significant new insights were enunciated. Measuring (with EM) the pad areas
adjacent to the synaptic cleft was mentioned.
9. Physiological verification of modeling in the intact brain
Whereas recording from brain slices, either by electrodes or Ca++ imaging, is
well established, neither technique translates to the interior of the intact
animal. Nonetheless we anticipate that imaging with cameras having frame times
of 1ms or less will open up more of the physiology to parallel inspection. Nanotechnology
may be able to provide us with more tools. For instance, a new generation of
electrode arrays based on carbon nanotubes might allow recording at finer spatial
resolution.
10. What can we do with the connectivity data once we get a full set?
We can then attempt to (1) model the development of the brain at the level of
its basic modules (currently not a focus of the modeling community), (2) identify
normal and pathological neural development (disease-related usage of the data),
and (3) gear up for a larger international program to simulate brain function.
Better liaison with animal behaviorist will be essential.
Participants:
Aravind Aluri, Texas A&M University
Dave Beeman, University of Colorado
Brad Busse, Texas A&M University
Yoonsuck Choe, Texas A&M University
Simon O’Connor, University of Cardiff
Juan Gomez-Molina, University of Texas at San Antonio
Greg Hood, Pittsburg Supercomputing Center
Heejin Lim, Texas A&M University
Alexander Maye, Zuse Institute Berlin
Bruce McCormick, Texas A&M University
Tom Morse, Yale University
Antonio Rogue, University of Sao Paulo
The discussion about the best approaches to optimization of neural
models centered around four main themes:
Perhaps the most important issue in optimization is selection of an
appropriate goodness-of-fit function. It is critical that the
function quantification capture the error between model and
experimental data in a meaningul way, that the function has a
minimum, and that there is a path to that minimum. Without an
appropriate goodness-of-fit function, the optimization can be doomed
before it starts.
Selection of an appropriate optimization method is very much problem
dependent. Genetic algorithms are useful for lots of parameters,
when searching over a discrete, coarse scale to find an approximate
minimum. However, GAs have many different variations which can
greatly affect the effectiveness of the search, and can have some
trouble identifying a global minimum in a continuous space.
Simulated annealing, on the other hand, can be quite useful for
finding the minima of a continuous space relatively quickly, and is
much easier to use than GAs. Gradient descent methods often get
stuck in local minima, so are often ineffective for neural models
which are highly nonlinear.
Because optimizations are generally performed over many parameters,
it is desirable to have the ability to identify sets of parameters
that are (nearly) independent from each other. The correlation
between parameters throughout the search space defined by the
goodness-of-fit function can offer some clues to this problem, but
at this stage the separation of independent variables it is still
very much an art.
A final issue is to make standard optimization tools more accessible
to the general user of neural simulators. This can best be done by
improving the documentation and online tutorials of the optimization
tools available in GENESIS and NEURON, and by encouraging advanced
users to share their goodness-of-fit functions and optimization code
with the general community.
Participating in the discussion:
Graham Cummins
Jeremy Edgerton
Michael Hines
Bill Holmes
Dieter Jaeger
Arnd Roth
Ruggero Scorcioni
Horatiu Voicu
Christina Weaver
For the purposes of keeping up-to date on the goings on of
computational neuroscience it has been agreed there is no substitute
to attending WAM*BAMM, but we feel there is a need for a more
persistent tool to increase communication between members of the
computational neuroscience community. The need for improved
communication tools became clear during our discussion, as quickly
realized resources for computational neuroscientists are scattered
about the web. We propose a website for the World Association of
Modelers for the purpose of increasing the flow of communication
between researchers, and as a resource to assist those in related
fields to join our effort.
One major drawback to the websites currently available is the
irregular updating. For example, the Neuroinf.org website includes
no mention of WAM*BAMM, or many of the software packages used by the
community. We propose a website based on wiki technology, which
allows users to edit the content of the site. In this way,
development of the website does not depend on a single person,
stopping when that particular graduate student moves on. Instead,
as long as there is a healthy and active community of modelers using
the website, additions will be made constantly and organically.
The content of the website will include the obvious elements,
compilations of software and summaries of their capabilities,
information on labs and researchers in the field, funding info,
meetings, and announcements. Additional features we desire include
an area for informal discussion about methods, tools to aid
referencing articles in journal databases, and tutorials (such as
the introductory tutorial given by Dave Beeman at WAM*BAMM). To
ease the pains of new researchers, either students or established
researchers from other fields, rather than simply linking to
external websites, deep-linking to documents within thier websites
which are particularly useful, such as to the Book of Genesis or
Neuron tutorials.
We would like to continue discussion on this topic, and are willing
to participate in its development. The resources needed to
implement this project are as follows:
-An initial bolus of work setting up the back-end technology.
-Seed articles constructed to reach a critical mass for usefulness.
-Resources for continued hosting.
-Promotion of the website to the computational neuroscience community.
Respectfully
Todd, Armen, and Padraig