NeuroMD SDK Manual

From Neurotech Software Development Kit
Revision as of 03:53, 22 August 2018 by NateColeman (talk | contribs) (Quickstart)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

This section contains information about software development kit required for using NeuroMD devices in user applications.

Introduction

NeuroSDK as a part of Callibri and Brainbit products is a set of libraries for different operating systems and platforms providing full set of tools needed for configuring and receiving data from Callibri and Brainbit devices. This software development kit consists of portable core written on C++11/14 and a set of wrappers for different languages and platforms: Java for Android operating system, Objective-C++ for iOS/MacOS, C# for Windows, Python for different platforms. Also, the core could be used for making cross-platform applications in C++ using different compilers and IDEs.

NeuroSDK is intended to be used by developers of different systems which requires biopotential data in conjunction with information about body movements and respiratory activity. NeuroSDK also provides tools for electrostimulation using Callibri EMS device.

Quickstart

To get started with software development kit you need to go through several steps which could vary in details for different platforms, but the essential sequence is constant. You may download and run some samples to get started.

Download and link

First of all download actual libraries for your platform and link them to your project. You could find detailed tutorial for each plantform on page Binaries and downloads. SDK limitations and features for BLE device communications on different platforms are also described there. Once you've downloaded and linked binaries to your project you may start using NeuroMD SDK classes to work with Callibri and Brainbit devices.

Scan for devices

To find devices you need to cteate DeviceScanner object, subscribe DeviceFound event and call startScan method. Using Java for Android you should pass application Context object to its constructor (see "Java" tab). If you're making your application using C++ you should call factory method createDeviceScanner to get scanner object.

#include "device_scanner/scanner_factory.h"
#include "device/param_values.h"

int main(int argc, char *argv[]) {
    auto scanner = Neuro::createDeviceScanner();
    scanner->subscribeDeviceFound([](auto&& device_ptr){
        onDeviceFound(std::forward<decltype(device_ptr)>(device_ptr));
    });
    scanner->startScan(0);//zero timeout for infinity
}
#include "cscanner.h"
#include "cdevice.h"
#include "cparams.h"
#include "sdk_error.h"

int main(int argc, char* argv[]) {
    DeviceScanner *scanner = create_device_scanner();
    scanner_set_device_found_callback(scanner, &on_device_found);
    scanner_start_scan(scanner, 0);//zero timeout for infinity
    scanner_stop_scan(scanner);
    scanner_delete(scanner);
}
var scanner = new DeviceScanner();
scanner.DeviceFound += Scanner_DeviceFound;
scanner.StartScan();
mScanner = new DeviceScanner(getApplicationContext());
mScanner.deviceFound.subscribe(new INotificationCallback<Device>()
{
        @Override
        public void onNotify(Object o, Device device)
        {
            //working with found device
            onDeviceFound(device);
        }
});
mScanner.startScan(0); //zero is for infinity

Read device name and address and connect to it

Basicly you should read device parameters only if device is in "Connected" state, but parameters "Name", "Address" and "State" could be read on "Disconnected" device. To read params call readParam method passing appropriate ParamName parameter to method in case of Java or with template parameter of type Parameter in case of C++.

template <typename T>
void onDeviceFound(T&& device_ptr){
    auto deviceName = device_ptr->readParam<Neuro::Parameter::Name>();
    auto deviceAddress = device_ptr->readParam<Neuro::Parameter::Address>();
    auto deviceState = device_ptr->readParam<Neuro::Parameter::State>();

    using device_t = typename std::remove_reference_t<decltype(device_ptr)>::element_type;
    auto sharedDevice = std::shared_ptr<device_t>(std::forward<T>(device_ptr));
    if (deviceState != Neuro::DeviceState::Connected) {
		connectDevice(sharedDevice);
    }
}

template <typename T>
void connectDevice(T&& device_ptr){
    using device_t = typename std::remove_reference_t<decltype(device_ptr)>::element_type;
    auto weakDevice = std::weak_ptr<device_t>(device_ptr);
    device_ptr->setParamChangedCallback([weakDevice](auto param){
        if (param == Parameter::State){
            auto device = weakDevice.lock();
            if (device != nullptr) {
                auto state = device->readParam<Parameter::State>();
                if (state == Neuro::DeviceState::Connected) {
                    readDeviceFeatures(device);
                }
            }
        }
    });
    device_ptr->connect();
    FoundDevices.push_back(device_ptr);
}
void on_param_changed(Device *device, Parameter param) {
    if (param == ParameterState) {
        DeviceState state;
        device_read_State(device, &state);
        if (state == DeviceStateConnected) {
            printf("Device connected");
            on_device_connected(device);
        }
    }
}

void on_device_found(Device *device) {
    char nameBuffer[128];
    device_read_Name(device, nameBuffer, 128);

    DeviceState state;
    device_read_State(device, &state);

    char addressBuffer[64];
    device_read_Address(device, addressBuffer, 64);

    device_subscribe_param_changed(device, &on_param_changed);
    if (state != DeviceStateConnected) {
        device_connect(device);
    }
    else {
        on_device_connected(device);
    }
}
private static void Scanner_DeviceFound(object sender, Device device)
{
    (sender as DeviceScanner)?.StopScan();
    device.ParameterChanged += Device_ParameterChanged;
    var deviceName = device.ReadParam<string>(Parameter.Name);
    var deviceAddress = device.ReadParam<string>(Parameter.Address);
    var connectionState = device.ReadParam<DeviceState>(Parameter.State);
    if (connectionState != DeviceState.Connected)
    {
        device.Connect();
    }
    else
    {
        OnDeviceConnected(device);
    }
}
    private void onDeviceFound(final Device device)
    {
        String deviceName = device.readParam(ParameterName.Name);
        String deviceAddress = device.readParam(ParameterName.Address);
        device.parameterChanged.subscribe(new INotificationCallback<ParameterName>()
        {
            @Override
            public void onNotify(Object o, ParameterName parameterName)
            {
                if (parameterName == ParameterName.State){
                    DeviceState state = device.readParam(ParameterName.State);
                    if (state == DeviceState.Connected){
                        runOnUiThread(new Runnable()
                        {
                            @Override
                            public void run()
                            {
                                onDeviceConnected(device);
                            }
                        });
                    }
                }
            }
        });
        device.connect();
    }

Explore device parameters, channels and commands

Now you could get information about device features: channels, commands and parameters. All communications with device modules are performed through these three substancies.

template <typename T>
void readDeviceFeatures(T&& device_ptr){
    auto commands = device_ptr->commands();
    for (auto& cmd : commands){
        auto cmdName = Neuro::to_string(cmd);
    }

    auto params = device_ptr->parameters();
    for (auto& paramPair : params){
       auto paramName = Neuro::to_string(paramPair.first);
       auto paramAccessMode = Neuro::to_string(paramPair.second);
    }

    auto channels = device_ptr->channels();
    for (auto& channel : channels){
       auto channelName = channel.getName();
    }
}
void on_device_connected(Device* device) {
    CommandArray commands;
    if (device_available_commands(device, &commands) == SDK_NO_ERROR) {
        printf("Device can execute:\n");
        for (size_t i = 0; i < commands.cmd_array_size; ++i) {
            Command cmd = commands.cmd_array[i];
            char commandName[32];
            command_to_string(cmd, commandName, 32);
        }
        free(commands.cmd_array);
    }

    ParamInfoArray parameters;
    if (device_available_parameters(device, &parameters) == SDK_NO_ERROR) {
        printf("Device has parameters:\n");
        for (size_t i = 0; i < parameters.info_count; ++i) {
            ParameterInfo paramInfo = parameters.info_array[i];
            char paramName[32];
            parameter_to_string(paramInfo.parameter, paramName, 32);
            char access[32];
            parameter_access_to_string(paramInfo.access, access, 32);
        }
        free(parameters.info_array);
    }

    ChannelInfoArray channels;
    if (device_available_channels(device, &channels) == SDK_NO_ERROR) {
        printf("Device has channels:\n");
        for (size_t i = 0; i < channels.info_count; ++i) {
            ChannelInfo channelInfo = channels.info_array[i];
        }
        free(channels.info_array);
    }
}
private static void OnDeviceConnected(Device device)
{
    Console.WriteLine();
    Console.WriteLine("Device can execute:");
    foreach(var cmd in device.Commands)
    {
        Console.WriteLine($"    -{cmd.ToString()}");
    }
    Console.WriteLine();

    Console.WriteLine("Device has parameters:");
    foreach (var paraminfo in device.Parameters)
    {
        Console.WriteLine($"    -{paraminfo.Parameter} {{{paraminfo.Access}}}");
    }
    Console.WriteLine();
    
    Console.WriteLine("Device has channels:");
    foreach (var channel in device.Channels)
    {
                Console.WriteLine($"    -{channel.Name}");
    }
    Console.WriteLine();
}
    private void onDeviceConnected(Device device)
    {
        Parameter[] deviceParams = device.parameters();
        for (Parameter param : deviceParams)
        {
            String paramName = param.getName().toString();
            String accessMode = param.getAccess().toString();
        }
    
        Command[] deviceCommands = device.commands();
        for (Command cmd : deviceCommands)
        {
            String cmdName = cmd.toString();
        }
    
        ChannelInfo[] deviceChannels = device.channels();
        for (ChannelInfo channel : deviceChannels)
        {
            String channelName = channel.getName();
        }

        subscribeBatteryLevel(device);
    }

Acquiring signal data, battery charge etc.

All devices provides information through various channels. For example, to obtain information about signal use Signal Channel class. It requires Device object as constructor parameter

Device abstraction

Every physical device, Callibri or Brainbit, is represented in NeuroSDK as Device abstraction which has state and set of possible control actions. Device state (should not be confused with connection state - one of device parameters) is characterized by parameters of device , which could be read with special function call readParam. Control actions are presented as set of possible commands and set of writable parameters. Commands could be executed by execute function, parameters could be set with setParam function. Device also has separate functions to control connection state. You can get list of all supported parameters by calling parameters member function. To get list of supported commands call commands member function. More information about these functions available on appropriate Device pages for different languages.

namespace Neuro {

class Device final {
private:
    std::unique_ptr<DeviceImpl> mImpl;

public:
    ~Device();

    void connect();
    void disconnect();
    std::vector<ChannelInfo> channels() const;
    std::vector<Command> commands() const;
    std::vector<ParamPair> parameters() const;
    bool execute(Command);
    void setParamChangedCallback(std::function<void(Parameter)>);

    template <Parameter P>
    typename ParamValue<P>::Type readParam() const;

    template <Parameter P>
    bool setParam(typename ParamValue<P>::Type value);

private:
    Device(std::unique_ptr<DeviceImpl>);
};

bool checkHasChannel(const Device &, const ChannelInfo &);
bool checkHasCommand(const Device &, Command);
bool checkHasParameter(const Device &, Parameter);
std::size_t countChannelsWithType(const Device &, ChannelInfo::Type);
ParamAccess getParameterAccess(const Device &, Parameter);

}
typedef struct _Device Device;

int device_connect(Device *);
int device_disconnect(Device *);
void device_delete(Device *);
int device_available_channels(const Device *, ChannelInfoArray *);
int device_available_commands(const Device *, CommandArray *);
int device_available_parameters(const Device *, ParamInfoArray *);
int device_execute(Device *, Command); 
int device_subscribe_param_changed(Device*, void(*)(Device*, Parameter));
namespace Neuro
{
    public class Device
    {
        public event EventHandler<Parameter> ParameterChanged;

        public IEnumerable<ChannelInfo> Channels { get; }
        public IEnumerable<Command> Commands { get; }
        public IEnumerable<ParamInfo> Parameters { get; }

        public void Connect()
        {
            SdkError.ThrowIfError(device_connect(DevicePtr));
        }

        public void Disconnect();
        public void Execute(Command command);
        public T ReadParam<T>(Parameter parameter);
        public void SetParam<T>(Parameter parameter, T value);
    }
}
public class Device {
    public final SubscribersNotifier<ParameterName> parameterChanged;

    public native void connect();
    public native void disconnect();
    public native ChannelInfo[] channels();
    public native Command[] commands();
    public native Parameter[] parameters();
    public native boolean execute(Command cmd);
    public native <ParamType> ParamType readParam(ParameterName param);
    public native boolean setParam(ParameterName param, Object value);
}

Device commands

Whole list of device commands consists of:

1. StartSignal command
2. StopSignal command
3. StartResist command
4. StopResist command
5. StartMEMS command
6. StopMEMS command
7. StartRespiration command
8. StopRespiration command
9. StartStimulation command
10. EnableMotionAssistant command
11. FindMe command

Device parameters

Set of command for specific device depends on which modules does device have. This is also true for parameters set. All possible parameters are listed below:

1. Name,
2. State,
3. Address,
4. SerialNumber,
5. HardwareFilterState,
6. FirmwareMode,
7. SamplingFrequency,
8. Gain,
9. Offset,
10. ExternalSwitchState,
11. ADCInputState,
12. AccelerometerSens,
13. GyroscopeSens,
14. StimulatorState,
15. MotionAssistantState,
16. StimulatorParamPack,
17. MotionAssistantParamPack

Device could be configured by setting these parameters, but whether each parameter could be set or not is represented by parameter access modifier:

1. Read,
2. ReadWrite,
3. ReadNotify

Parameters could be readonly, for some of them device provides notification mechanism. Parameters with ReadWrite notifier could be set from user code.

Channel library

All NeuroMD devices provides biopotential signals and telemetry information through channels. Channel set of device depends on device modules. Information about channels could be obtained by calling channels method of device class.

SignalChannel

Built-in channel wich provides access to signal data received from device

private static void OnDeviceConnected(Device device)
{
     var frequency = device.ReadParam<SamplingFrequency>(Parameter.SamplingFrequency);
     if (frequency != SamplingFrequency.Hz125)
     {
         device.SetParam(Parameter.SamplingFrequency, SamplingFrequency.Hz125);
     }
     var signalChannel = new SignalChannel(device);
     signalChannel.LengthChanged += SignalChannel_LengthChanged;
     device.Execute(Command.StartSignal);
}

private static void SignalChannel_LengthChanged(object sender, int length)
{
     var duration = (double) (length) / 125;
     if (length > 125)
     { 
         var data = (sender as SignalChannel)?.ReadData(length - 125, 125);//read last second of signal on 125 Hz sampling frequency
     }
}
private void createSignalChannel(Device device){
    ChannelInfo[] deviceChannels = device.channels();
    for (ChannelInfo info : deviceChannels) { //creating spectrum channels for each signal channel present in device
         if (info.getType() == ChannelType.Signal) {
             final SignalChannel signalChannel = new SignalChannel(device, info);
             signalChannel.dataLengthChanged.subscribe(new INotificationCallback<Long>() {
                 @Override
                  public void onNotify(Object sender, Long length) {
                      long dataLength = 1024;
                      if (length > dataLength){
                          double[] signalData = signalChannel.readFast(length - dataLength, dataLength); //read last 1024 signal samples
                      }
                  }
            });
    }
}

BatteryChannel

Built-in channel containing battery charge data

private static void OnDeviceConnected(Device device)
{     
      var batteryChannel = new BatteryChannel(device);
      batteryChannel.LengthChanged += BatteryChannel_LengthChanged;
}

private static void BatteryChannel_LengthChanged(object sender, int length)
{
      var batteryLevel = (sender as BatteryChannel)?.ReadData(length - 1, 1)[0] ?? 0;
}
private void subscribeBatteryLevel(Device device){
    final BatteryChannel batteryChannel = new BatteryChannel(device);
    batteryChannel.dataLengthChanged.subscribe(new INotificationCallback<Long>() {
       @Override
       public void onNotify(Object sender, Long length) {
            int batteryLevel = batteryChannel.readData(length - 1, 1)[0];
       }
    });
}

ConnectionStatsChannel

ElectrodesStateChannel

MEMSChannel

OrientationChannel

RespirationChannel

SpectrumChannel

Programm channel providing spectrum data for specified signal channel

private void createSpectrumChannel(Device device){
    ChannelInfo[] deviceChannels = device.channels();
    for (ChannelInfo info : deviceChannels) { //creating spectrum channels for each signal channel present in device
         if (info.getType() == ChannelType.Signal) {
             SignalChannel signalChannel = new SignalChannel(device, info);
             final SpectrumChannel spectrumChannel = new SpectrumChannel(signalChannel);
             spectrumChannel.dataLengthChanged.subscribe(new INotificationCallback<Long>() {
                 @Override
                  public void onNotify(Object sender, Long length) {
                      long dataRangeLength = 1024;
                      if (length > dataRangeLength){
                          double[] spectrumData = spectrumChannel.readFast(length - dataRangeLength, dataRangeLength); //calculating spectrum for last 1024 signal samples
                      }
                  }
            });
    }
}

Base channel

All channel classes which device has are inherited from BaseChannel abstract class/interface. Custom channels (channels which do not present in device, also user-created) could be inherited from BaseChannel, but it is not necessary. Representing device information by channels gives easy way to process heterogeneous information in one manner. For example, it is easy to plot function of battery charge, biosignal, and connection statistics depend on same time scale.

namespace Neuro {

template <typename DataType>
class BaseChannel {
public:
    using data_container = std::vector<DataType>;
    using length_callback_t = std::function<void(data_length_t)>;
    using length_listener_ptr = ListenerPtr<void, data_length_t>;

    BaseChannel(ChannelInfo &&info) noexcept : mInfo(std::move(info)) {}
    BaseChannel(const ChannelInfo &info) : mInfo(info) {}
    virtual ~BaseChannel() = default;

    ChannelInfo& info() noexcept {
        return mInfo;
    }

    virtual length_listener_ptr subscribeLengthChanged(length_callback_t callback) noexcept = 0;
    virtual data_container readData(data_offset_t, data_length_t) const = 0;
    virtual data_length_t totalLength() const noexcept = 0;
    virtual data_length_t bufferSize() const noexcept = 0;
    virtual sampling_frequency_t samplingFrequency() const noexcept = 0;
    virtual void setSamplingFrequency(sampling_frequency_t) = 0;
    virtual std::weak_ptr<Device> underlyingDevice() const noexcept = 0;

protected:
    ChannelInfo mInfo;
};

}
public abstract class BaseChannel<SampleType> {
    public final SubscribersNotifier<Long> dataLengthChanged = new SubscribersNotifier<>();
    public abstract ChannelInfo info();
    public abstract SampleType[] readData(long offset, long length);
    public abstract long totalLength();
    public abstract long bufferSize();
    public abstract float samplingFrequency();
    public abstract void setSamplingFrequency(float frequency);
    public abstract Device underlyingDevice();
}

Devices versions

Callibri and Brainbit devices could have different sets of modules and functions depend on purpose of specific device.

Currently NeuroSDK supports three types of devices:

• Callibri (for electrophysiology);
• Callibri EMS;
• Brainbit.

In this section all possible compositions of modules or functions are listed

Callibri

Since Callibri device is positioned as universal biopotential measuring device it has modules to provide as much information as possible.

All posible modules of Callibri device are:

Base Callibri module

Base module is responsible for common Callibri device configuration and BLE configuration.

Module parameters:

1. Name;
2. State;
3. Address;
4. SerialNumber;
5. FirmwareMode;

Signal module

This module controls wide range of bio signals acquisition and its parameters such as sampling frequency, gain etc.

Module parameters:

1. HardwareFilterState;
2. SamplingFrequency;
3. Gain;
4. Offset;
5. ExternalSwitchState;
6. ADCInputState;

Respiration module

Respiration module controls special signal acquisition - respiration characteristic.

Module parameters:

1. ExternalSwitchState;

Micro electro-mechanical system (MEMS) module

This module is responsible for orientation and acceleration tracking of device.

Module parameters:

1. AccelerometerSens;
2. GyroscopeSens.

Most commonly Callibri device are supplied with all of these modules. Modules could be configured through device parameters. It is also possible to execute set of module-specific commands. List of supported commands and parameters could be acquired from device by appropriate calls to device library.

Callibri EMS

Callibri EMS device is positioned as smart electromyostimulation device and provides ability to stimulate muscles based on orientation and acceleration tracking data. It includes:

Base Callibri module

Base module is responsible for common Callibri device configuration and BLE configuration.

Module parameters:

1. Name;
2. State;
3. Address;
4. SerialNumber;
5. FirmwareMode;

Micro electro-mechanical system (MEMS) module

This module is responsible for orientation and acceleration tracking of device.

Module parameters:

1. AccelerometerSens;
2. GyroscopeSens;

Stimulation module

Special module for Callibri EMS which provides output with stimulation current pulses for miostimulation.

Module parameters:

1. StimulatorState;
2. MotionAssistantState;
3. StimulatorParamPack;
4. MotionAssistantParamPack.

Brainbit

Brainbit is more simplier than Callibri device and has only two modules

Base Brainbit module

Base module is responsible for connection establishing through BLE and for device state and mode control.

Module parameters:

1. Name;
2. State;
3. Address;

Signal module

Signal module is responsible for acquisition of EEG signal from 4 device electrodes and for measuring electrodes resistance.

Module parameters:

1. SamplingFrequency;
2. Gain;
3. Offset;

There are no modifications for Brainbit device, so basically it has fixed set of commands and parameters. Anyway, we are moving forward and add new post analytic functions for Brainbit data.