Extract log details tutorial

[Don Fukushima]

I did not find anything like a step-by-step guide for extracting the log data from the SBMS0 but did find the clipped text below from the manual. That mostly explains the data but not really a prototype of process for extracting the data. There is mention of  uncompressed data and formatted in .csv  but short on details how to make that file available on a connected device like a laptop. 

Clipped

The SBMS has an internal 128Mbit (16Mbyte) SPI Flash memory that can log data at a fixed 2 minute interval. The memory is capable of storing a full year (12months) of data in a circular type buffer so you always have the last 12 months worth of data in that internal Flash memory.

In order for the data logging to start you will need to set the Date and Time and you will notice the small “mem” symbol in the lower right side of the SBMS LCD being highlighted.

The data is also stored compressed in the memory same base 91 compression mentioned in the WiFi chapter and contains the data sent also over WiFi in the sbms variable.

Here is an explanation of what is contained there and what it is logged in the internal memory.

var sbms="YMDHMS%%C1C2C3C4C5C6C7C8ITET+BATPV1PV2EXTAD2AD3AD4ht1ht2ERR"

Y-year last two digit (limited to 2090); ( sorry if you read this in 2091 I will not expect a software update :) ).

M-month

D-day

H-hour

M-minute

S-seconds

%% - SOC 0 to 100%

C1 to C8 - cell voltage in mV

IT and ET - internal and external temperature in hundreds of a degree 0 is -45C and 1449 is 99.9C so you need to subtract 450 from the result and divide by 10 (example 689 as result of the decompression will be (689-450)/10=23.9C) + - sign for battery charging (+) and discharging (-) used as it is no compression.

BAT - Battery current in mA from -750000mA to +150000mA

PV1 - PV1 input current in mA

PV2 - PV2 input current in mA

EXT - Ext Load current (the use of 10A up to 500A 75mV external current shunt is possible on both SBMS40 and SBMS120 see page 13)

AD2 and AD3 - user available ADC inputs the value is the digital output of the ADC.

ht1, ht2 - reserved for power output of the DMPPT450

ERR - this is error code it will contain the information referring to the error flags on the SBMS.

There are 15 Flags (see monitoring page3 on SBMS) not all flags represent an error some are representing normal operation and if you consider this a 15bit binary number each bit representing one flag with “1” highlighted and “0” not highlighted and you start from the top left with OV flag and continue to right and then down to the last flag DFET and consider the OV as the least significant bit and DFET most significant bit then you get this binary number and convert to decimal that will be the value contained in this ERR Status and you can also see this number on the SBMS LCD just under the battery SOC value and also displayed as Status: in the html page.

Uncompressed data and formatted in .csv as it was on SBMS4080 will look like this

Y ,M ,D ,H ,M ,S ,SOC,cel1 ,cel2 ,cel3 ,cel4 ,cel5 ,cel6 ,cel7 ,cel8 ,IT ,ET ,-Batt , PV1 , PV2 ,ExtLoad, ADC2, ADC3, ADC4, heat1 , heat2 , ERR

00,01,31,22,53,47,100,3414,3416,3444,3433,3426,3419,3420,3380,000,000,-002000,000000,000000,000000,000000,000000,000000,000000,000000,000000

This is the javascript function for decompression (you will find this inside the SBMS webpage) function dcmp(p,s,d){xx=0; for (z=0;z<s;z++){xx =

xx + ((d.charCodeAt((p+s-1)-z)-35)*Math.pow(91,z));}return xx;}

where

p - location of the data in the string.

s - size of the data (example year is 1 cell voltage is 2 and PV1 current is 3)

d - the name of the variable in this case sbms.

You can download the compress raw data available in the internal memory over UART by going to Device Settings menu and Internal data log sub-menu were you can select the month you want to download.

Before that set the UART to ON and the baud rate you prefer in the Device settings and connect the USB port to your computer if you have the WiFi/USB extension board or if not you can use the UART TX line from SBMS0 16pin connector but make sure to add some sort of optical or galvanic isolation if your computing device is supplied from same main Lithium battery that SBMS0 protects.

On your computing device have some sort of serial data logger that should collect all the data that will be transmitted when you push the Start Download data button.

Below you see an example of one hour of compressed data made of 30 sets of data since it will log each 2 minutes and there are 744 hours in a full month and you can get why I can not list all in the manual. When saved as a text file it will be about 1.4MB of compressed data.

[Dacian Todea]

Don,

You have live data available over UART or USB (assuming SBMS0) and there is internal stored data that can also be downloaded over serial UART or USB (same thing on SBMS0).

What are you interested in exactly ?  The data no matter if live data or stored data that you download will be compressed and if you want to decompress you need to use the formula available in the user manual. 

[Bernd]

Hi Don,

do you have SBMS0 with wifi and can access the rawdata page from your laptop? If yes, this is the way to go if you are interested to monitor the current status of the bms.

If you wait to download data e.g. weekly and run a query in a database or excel, then you need to stick into the USB download.

Yours Bernd

[Don Fukushima]

I did read there are two ways to retrieve data: real time feed and dumping stored logged data. I want to get the logged data off the SBMS0.. 

The SBMS0 has wifi so I guess the way to do this is "then you need to stick into the USB download." That is the part I'm looking for a bit more detail. 

Things I know for sure

1) The SBMS0 and client device (laptop, tablet) are connected via wifi as the real time feeds shows up in the browser.

2) Based on the documentation, there is stored logged data in the SBMS0

3) Enough posts about the logged data being offloaded somehow.

That is where I am at. 

What is needed now to get access the log files on the device, select them, and transfer to the PC or tablet?

[Don Fukushima]

"there is internal stored data that can also be downloaded over serial UART or USB (same thing on SBMS0)."

Yes, this is what I think is the right solution. Is this documented what is needed to make this happen? Is the transfer done over wifi or is a cable needed that support UART or USB? thanbks

[Dacian Todea]

Don,

Over WiFi you can only get the live data + the graph for last 12h

The internal stored data that is for an entire year at 2 minutes interval can be downloaded only over USB/UART.

The way you do that is to connect your PC or tablet to USB on the SBMS0 and then start your preferred serial logger program so that is ready to receive data at the speed set in the settings while the SBMS0 is in the internal data log menu. From the menu select the month you want to download and then push the Start data download button and all data will be transferred about 1.4MB worth of compressed data that your serial program on the computer will save in to a file. You will need to repeat that for any other month you want to download. The SBMS0 will always store the data internally as long as you set the date and time and it will store the last 12 months worth of data. 

[Don Fukushima]

Which are the common serial loggers for Mac and Android? Sorry, this is a utility that I've not needed in the past. thanks

[Don Fukushima]

I found this link that talks about two MAC serial port tools

https://www.virtual-serial-port.org/articles/serial-port-monitor-mac/

Coolterm and Serial were two programs. anyone used these? 

[Dacian Todea]

Don,

I'm not familiar with mac but that looks like it should work as it allows you to log the data to a text file.

[Don Fukushima]

I was able to offload internal log file using  serial 2 app for Mac. 

For some reason the SBMS0 wifi is not working. I toggle the wifi on/off. it tries but then returns to wifi OFF state.

From the directions, it says the decompress function is accessed via the html page. Well, can't load that up so what other way can I feed this compressed file in and get uncompressed output file?

[Dacian Todea]

Don,

There is no decompression function just a formula that is in the user manual and the same formula is used by the HTML file to display the data but that file cannot decompress this long term data.

Your log file is to small and looks like just garbage so likely the SBMS0 was not set on USART mode and or the baud rate was set incorrectly.

So go to USART Setting set the USART data log to 1 then go to Baud rate and set that to 115.2K

Then after you did that go out of this menu and go in to internal data log menu and then go to your computer and select the same 115200bps as the baud rate having your serial loger ready to start logging all data that will start coming as soon as you push the Start data download button on the SBMS0

The data inside the file will need to look like this 

//ElectroDacus SBMS0   Month: 08 

 var h000="ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ"; 

 var h001="ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ"; 

 var h002="ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ"; 

 

This is just a small part of the file it will continue like this up to h743 meaning hour number 743 thus 744 hours total in a month.

Here you only see ÿÿÿÿ as data meaning there is no data as this was downloaded from a new SBMS0 so there was no logged data to show this will be replaced with real data once there was something logged.

Basically each hour contains 1800 characters groups of 60 so 30 of them representing a set of data at each 2 minutes.

[PeterBC]

Don,

Did you manage to get your data and decompress it ?
Has anyone a simple windows or similar method of decompressing this data. I havent connected to a serial port in 20 years and last data logger I ran was Crosstalk in the 80's. Late 80's was my last programming too in Turbo Pascal.  I gather I can get a physical USB cable and hack the end of it and somehow connect to that via a Windows PC ? 
Just to clarify the "Save Log" button that creates the 'live data' log, has how much and what interval data ? 12 hrs at X seconds ?
Its still growing by 41kbytes in 20mins after more than 24hrs. When will it stop growing and wrap ? Is it a separate storage area from the long term 12 month data ? Is that area accessible by any web page like the 'live data' is ?  

Is each 60 byte record the same as the 2min long term yearly log ? ( except each hr record has 30 x 60 chars (59 chars and a LF char ?)  ) 
I suppose I'll try to keep going the MQTT route and try to benefit from Soggy Paw's excellent work, but thats also a steep learning curve to get it configured. I have had the SBMS logging to Mosquito  but yet to get NodeRed doing anything sensible.  I noticed the quiesent current draw of the SBMS0 went from 22 ma to about 200ma with it logging to a Mosquito listener. 

Daican, it would be really nice if there was a simple button on the webinterface to download a decompressed asci file of the logged data .. I suspect you could do it trivially ? 
But I gather there isnt a simple way to field upgrade the 'firmware' on an SBMS0 ?

rgds

[Dacian Todea]

Peter,

I assume you have a new SBMS0 with WiFi/USB installed that means you have a mini USB port on the side of the SBMS0 so you just need a USB type A in to computer to mini USB in to SBMS0 nothing else or special just one of your old phone cables will work just fine.

That browser Save Log button just saves whatever it was stored while the browser window was open. So say you opened that browser window a half hour ago and then decided to push the Save Log button then data there is what happened in that half an hour the browser window was open and if data was updated every 3 seconds then it will be data at 3 second interval for the last half an hour or since you last pushed the Clear Log.

The max amount of data that can be stored that way will depend on your computer / browser settings as it utilizes the browser allocated temporary memory (how large is that will depend from one system to another).

Yes the 59 char are always the same thing explained in the manual starting with date and time followed by battery SOC the cell voltages and then currents and ending with error codes.

There is no way for the SBMS0 to use 200mA even with a 12V battery that will be over 2.4W and the SBMS0 can not use more than 1W peak when WiFi is ON.

The internal data log that one year at 2 minute interval can only be downloaded over USB thus web interface is not useful.  The web interface already decompress and displays all live data.

The compressed data is at least 2.5x smaller than simplest uncompressed csv thus the reason data is stored internally compressed so a full year can be stored and also transmitted compressed so it saves time and the decompression can be done outside with just a simple formula shown in the user manual.

The compression is sort of similar to hex vs dec is just that instead of base 16 to base 10 conversion a base 91 is used.

Intention was to build an application maybe even a web based one to decompress the internal data log data that is saved to a file in to a csv type file but I will need a few days (probably a full week) and never had that sort of time. Also anyone should be able to do that and it is external to SBMS0 so I was expecting someone will do that.  Also data can be saved compressed for when the tool to convert exists so nothing is lost.

As mentioned one month of data at 2 minutes interval contains about 1.4MB so converted to .csv should be at least around 4MB then that can be used to maybe build a graph or something like that.

But you will have 22320 data points in a month so I can imagine Excel or OpenOffice equivalent will struggle a bit with that amount of data points and that is just one month not a full year. 

[B0bD]

For whatever its worth, here is the approach I use.

It is only good for handling a download - it doesn't do anything about real time monitoring.

For the connection to the SBMSO, a standard USB charging cable will work.

The port appears in Device Manager as USB2.0 serial.

I didn't have the drivers, but once I found and installed them it all worked.

I use a port monitoring program called Free Serial Monitor - its a free download and while simple suits my purpose.

Set the rate to 115200 on both the SBMSO and the monitor.

Set a file name in the monitor, then start logging

On the SBMSO set the UART to start the download.  Takes a few minutes.

Open the file using Notepad  ++ - another free download, with better editing than Notepad.

The file needs to be rearranged a bit:

1. There will be rows filled with "yyyyyyyy........", There are for days without data, and it makes it easier if these are erased.
2. Search>> Find>> [ var h.*="]  ( that's space var space h .*=* )   Replace>>nothing             This erases the first few characters of each line - you don't need them
3. Search>>Find>> [ "; }  (that's space " ; space)  replace >>nothing           This erases un-needed trailing characters from each 59 character record
4. Search>>Find>>space   Replace with \n           This separates each 59 character record into a new line
5. Save the file
6. Import into excel.  This can be done directly into the spreadsheet prepared by DShowman (https://groups.google.com/g/electrodacus/c/R7vfB0lVzv0/m/uy7j1GQiCAAJ)
7. I find it easier to first import the data into a new Excel file:
   1. Open a blank worksheet
   2. Data>>Get data From Text/CSV
   3. Select the file saved above
   4. transform Data>>Split Column>>By Number of Characters>>1, repeatedly
   5. Close & Load - this puts the data into the blank worksheet
   6. Scroll down and see if there are any blank rows, or ones with inconsistent looking data - I just erase them
   7. Then cut the 59 columns by however many rows there are, and paste into the spreadsheet mentioned above.
   8. The Tab in the spreadsheet "Calculated" is the information you want.
8. I have noticed that sometimes there is a value missed, and it may or may not be worthwhile fixing it, depending on whether that data point is critical.

This looks complicated but actually only take a couple of minutes.

I am sure it can be cleaned up with macros, but for my purposes, which is to take a look at things only occasionally., it is not worth the time.

I couldn't find anything at this very low level when I was figuring out how to get at the data, so thought this might be useful.

[PeterBC]

Doh !  ... I'd been ignoring the USART discussions thinking I had to custon make a USB cable. Hadnt noticed the MicroUSB port there..

So I plugged in, got the USB 2.0 Serial device without driver in my Device Manager.

Did details and say the device ID .. did a search usb vid_1a86&pid_7523 driver windows 10
First hit was https://www.catalog.update.microsoft.com/Search.aspx?q=USB%5CVID_1A86%26PID_7523
First hit Download  worked (in Edge, Chrome wasnt playing ball for me).   Driver Model: USB-SERIAL CH340
Then extract the files from the .cab  to a local directory. Then update driver, browse to files worked and I had a COM4 USB serial port.
I then used Putty with serial port, COM4  and baud rate to 716800  (700k) and set 700k in the SBMS0. 
So its spitting our a record every 30secs ? 
But I then went to the downlod month and start and it spat out all the data.. 

Now need to get it or something to log that to a file.. 

var h693="▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

..snip .. ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒";

              va▒ h694="▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

..snip..

▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒";

.. I understand Daican now how the storage on your microcontroller is so limited so you had to do some tricks. Its a miracle you got the graphs up you do ! 

In my parallel work I've managed to get Soggy Paws MQTT and Node-Red installed and Sherry's json file up  modified for my 8 cells and basically working..
it too creates daily log files and summaries as CSV's.  Not sure that amount of overhead on my PC to run all the time is going to be worth the hassle.
I'm happy with the Wifi page for live and recent monitoring, just wanted to be able to review past history a bit. 

I might just be happy to dump this internal data log and have a poke in excel 

Thank.. 

[Dacian Todea]

You can set that interval down to 1 second from the default 30 seconds and the setting for that is in the USART settings menu on the SBMS0. You can select any value between 1 second up to 240 seconds with 1 second interval.

The SBMS0 has a separate 128Mbit (16MB) Flash memory where the 1 year data log is saved and the internal graphs shown on the SBMS0 for max 24h are stored in the Microcontroller internal RAM just 32KB.

You can use Putty to log to file.

[PeterBC]

I got  a 1.3x MB file from Putty for a month..

But it has some corruption in characters.. several 'greek' bits..  I did try at the default speed of 921.6k but thats a fractional decimal ? 943,718.4 speed which I cant put in Putty ?
Is speed even a thing for these 'virtual' USB serial ports ? Seems so as other values caused garbage. 

[Dacian Todea]

Try a much lower baud rate below 460.8kbps as data looks corrupt and from my tests on USB speed above 460.8kbps are not reliable.  The 921.6kbps is just default as that is what is used internally for the WiFi module ESP32 to communicate with the microcontroller but there is a short distance and neither the digital isolator or CH340 serial to USB is not involved.

[PeterBC]

I repeated at 500k USART speed and 512,000 speed in PUTTY and it was clean.. 

rgds.. Pete