Senseless

is it possible without additional Hardware?

senseless.png

A number of approaches are available in most modern environments that allow sensing of users, their actions, or potential interaction intent. Importantly, many of these techniques are possible without deploying additional (and unnecessary) hardware, forcing explicit interaction from users, or changing how they interface with a physical space.

Many of these inputs don’t provide significant value in isolation, however using a single platform to aggregate and correlate multiple sources can provide signifiant context on environments, as well as the people within them.

This is a brief rundown of a talk at Integrate Conference 2019 in Melbourne, Australia.

WiFi

Deployed in almost every space and provides an excellent location service for users that are connected to the network.

How it works

  • Three main techniques. Availability of these will be dependent on Access Points (APs) and associated wireless platform installed.

    • AP association - detects which access point a user is closest too. Very coarse accuracy. Almost useless.

    • Trilateration - looks at received signal strength indication (RSSI) across APs to provide an estimate of device location. Accuracy can get down to 2 - 4m (depending on environment)

    • Triangulation - uses Angle of Arrival (AoA). Super accurate (< 1m), but limited hardware support today.

  • Generally a single platform (Cisco CMX etc) will coordinate and aggregate this info from all APs, which can then be queried to provide either instantaneous device location, or in some cases location history.

What it provides

  • Location of portable devices

  • Location of users (via device association


Bluetooth

How it works

  • Very similar to WiFi - single nearby beacon detection, multiple beacon trilateration or triangulation

  • In most cases though the roles are flipped.

    • With wifi, the fixed access points are used to locate a device.

    • In BLE fixed beacons are used by a device to locate itself.

  • An app running on that device may then use this location context itself (e.g. for performing proximity dependent actions such as showing a contextual UI, or may report this back to a central system for location analytics).

What it provides

  • Location of mobile devices.


Optical Sensors

How it works

  • Many fixed cameras will provide on-device processing.

  • The capabilities of this can range from rudimentary motion / presence detection to AI based systems that provide object identification, or semantic information about what is currently in frame.

  • When edge processing is available it should always be preferred as it provides not only an efficient solution from an engineering standpoint, but also provides the greatest privacy as raw images are never sent to external systems.

What it provides

Capabilities will vary depending on hardware, however normally one or more of these will be provided via on-device image processing.

  • Motion detection (either in frame, or in parts of frame)

  • New object detection (background subtraction - again either for full frame, or within specific ‘zones’)

  • Luminosity (useful for both natural and artificial light sources - depending on environment and time of day this could be are the blinds open, are the lights on etc).

  • Face detection (is there a person in frame?)

  • Face counting (how many people are in frame?)

  • Tagging of objects in frame (there is a person / face at coords {x,y}, a laptop at {x,y} etc)


Ultrasonic Beacons

How it works

  • Two devices are used:

    • The emitter: this must be outputting audio at an appropriate volume level and frequency. 

  • The receiver: must have a microphone capable of capturing the emitter’s signal and the power to process this.

  • The emitter generates an audio signal (usually at frequencies > 18kHz) that includes encoded information, such as a unique ID.

  • Receiver then listens for this, and extracts the encoded information.

What it provides

  • Confirmation two devices are sharing the same physical space.

  • Room level device (/device owner) location when either the emitter or receiver is fixed in a known location.


Acoustic Signal Presence

How it works

  • DSPs, pre-amps and other audio equipment will generally provide signal presence indication or have the ability to add a signal gate that may be monitored.

  • Activation of this may be monitoring as an input for presence detection.

  • For this to be a useful indicator it will generally require smoothing (must remain active for x% time over y seconds) and latching (once activated, presence will indicate for z seconds). This may either be achieved by setting up the gate threshold and hold times if the gate is used for this monitoring only, or by smoothing instantaneous status externally.

What it provides

  • Basic occupancy sensing based on movements or speaking within a space.



Passive Infrared (PIR)

How it works

  • Everything (above absolute zero) emits some low level radiation.

  • PIR sensors detect IR levels across two beams and use to the difference between these signal to tell when an object moves from one to another.

  • Lenses are then used to split these two ‘sensor beams’ into multiple that cover the detection area.

  • This is why when you are sometimes sitting still in a room for an extended length of time the lights may automatically turn off. Detection is based around movement, rather than presence.

What it provides

  • Presence detection (normally room level granularity).


Carbon Dioxide

  • Part of HVAC, available via BMS

  • CO2 levels within a space indicate level of occupancy.


Door Interactions

  • From security system.

  • Simple open / close sensing.

  • User detection and instantaneous location from swipe card use.


Lighting Activation

  • Manual triggering of a light switch

  • Indicates a user is present within the space, or leaving a space.


Calendaring System

  • Eg Google Calendar or Microsoft Office

  • Provides intent of:

  • when spaces are planned for use

  • who is using them

  • what they are using them for