(WMSC’07)
Sentient computing has been a term mainly used
for location-aware computing and ubiquitous computing. However,
“sentient” literally means “aware” or more specifically “finely
sensitive in perception or feeling”. It seems to us that sentient
computing not only needs to be location-aware, but also identity-aware
and context-aware. Surveillance and security are probably the best
applications for sentient computing because here a surveillance or
security system needs to be “finely sensitive in perception or feeling”
for locations, identities and environments, using multimodal sensors.
Then, what will be the impact when we introduce "multimodal sentient
computing" to surveillance and security applications?
Results reported for multimodal
fusion in sentient computing systems can be difficult to compare due to
the differences in the underlying sensor systems and deployment
architecture. Many problems in computer vision and machine learning
have greatly benefited from the availability of standardized data sets
and benchmarking procedures. Would it be desirable to create a
standardized evaluation framework for multimodal sentient computing,
perhaps along the lines of the "Performance Evaluation of Tracking and
Surveillance" workshops series?
How useful and necessary is it
to introduce levels of abstraction away from the hardware level? From
an architectural point of view, it is desirable to only present
applications and users with the system's best guess as to the
locations, identities and states of objects. However, in practice one
often finds that the particular error characteristics and properties of
each sensory modality have a profound impact on multimodal sentient
computing application design, so where can abstractions actually get in
the way?
As a related issue, when do we
want to perform sensor fusion? Applications may have their own
requirements regarding accuracy, fault sensitivity, and tolerance to
false positives. Some multimodal sentient computing systems consider
sensor fusion to occur in a middleware layer, but we may need to
provide richer APIs that allow sensor fusion to be driven by
application
demands. To what extent is
sentient computing a sensor fusion problem as opposed to an inference
(artificial intelligence) problem? Even we had perfect sensors capable
of measuring any physical characteristic of the environment with
perfect accuracy and reliability, how much closer would we be to
maintaining an accurate world model?
There are also a range of issues
to do with sentient computing systems in general, including:
(1). To what extent can user
location/identity/context data be used to create models of the
environment?
(2). How can user
location/identity/context data be used to monitor the status of the
world model?
(3). How can sensor system
reliability and accuracy be learned from observational data? How can we
automatically detect sensor faults and compensate for them?
(4). What are the privacy
implications with releasing varying levels of location/identity/context
data?