[vz-dev] API

Tue Jul 20 00:17:08 CEST 2010

Hi Steffen,

On 07/19/2010 05:47 PM, Steffen Vogel wrote:
> Hi,
>
> Am Sonntag, den 18.07.2010, 21:11 +0200 schrieb Justin Otherguy:
>> so, basically we have two parts:
>> - header info: mime-type + protocol version 1) + access token
>> - request paramaters:
>>    - identification of the sensor and controller:
>>      - Flukso:
>>        - "sensor" (references the ucid, correct?)
>>        - there seems to be no keyword to reference the controller / flukso meter, yet - correct?
>>      - VZ:
>>        - "ucid" identifies the sensor ->  I suggest switching to "sensor" here; like flukso and more readable than "ucid"
>>        - "uuid" identifies the controller ->  I suggest switching to "meter" here; this is more readable than "uuid" and more precise than "user id"
>
> Actually, we do not use ucids or uuids. These are only identifiers to
> identify a channel (aka meter, sensor) or a user.
>
> Some weeks ago, i talked to Florian Ziegler. We came to an understanding
> of the following definitions:
> http://de.volkszaehler.wikia.com/wiki/Entwicklung/Definitionen.
>
> As Volkszaehler seems to become a more generally oriented framework, we
> do not only have meters.
> Our current implementation knows about the general term "channel". For
> which we have implemented two diffrent channel types:
> - meters (power, water, gas, etc.)
> - sensors (temperature, pressure, humidity, etc.)
>

I'd say both are sensors. The meter type will integrate it's readings 
while the 'other one' (I can't seem to find a suitable name) will make 
use of the instantaneous readings.

>>    - reading/count: we've discussed that one already; should not be the number of pulses since the last request, but the total number of requests; Steffen asked a very good question here: what happens after a reboot of the controller or any other event that would cause the counter to restart at 0: I don't think it's a big issue to detect this and take care of it while reading the values from the db. I can only think of one tricky situation: the interval between two data points in the diagram is greater than the interval between two reboots/counter resets; I don't think that should be a real world issue, should it?
>>
>>>> And how do we want to save this value (EEPROM vs. RAM?).
>> speaking of AVR net io: we'll have to find out - I don't know yet; EEPROM would be the place, I suppose (RAM: doesn't survive a reboot; SD: in most cases not available);
>> Flukso: shouldn't be a big deal :-)
>
> Ok, this storage question seems to be solved.
>
>>    - "unit":
>>      I just understood the different approaches we had; so far we (VZ) are transmitting pulse counts that are calculated into consumptions later on; that's why we need to know the meter's resolution on the server side; I can't see any disadvantage of switching to the flukso approach here: transmitting the consumption together with the unit. Can someone else? Otherwise: let's get rid of ours!
>
> I agree to Justins argumentation. But that would need an configuration
> on the controller side. Without an Webfrontend this could become quite
> hard.
> The setup should be as easy as possible.

The guys at Fraunhofer are implementing a wizard on top of LuCI with 
configuration error checking to alleviate installation problems.

>
>>    - "interval":
>>      we don't have that at all so far; what's the idea behind transmitting that as well?
>
> We also have an interval parameter. In our backend, its called "groupBy"
> and should take one of these values: year, month, week, day, hour,
> minute or second.
>
> Attention Justin, i assume you are mixing both API parts.
> The API call you pasted above is just for Frontend->Backend.
>
>> remarks:
>> 1) in our api reference [1] we currently have the version information as one of the regular request parameters; I don't mind switching to that variant; we'll be using http headers anyway
> Im a little bit unsatisfied with mixing HTTP Headers and GET Parameters.
> Why dont we use GET parameters at all? Sometimes it could be quite hard
> to send custom HTTP headers. Especially when we have to use an framework
> in the future which doesn't support this (Ethersex, Chumby etc..).
> Or, whats about debugging capabilities? Wasn't that easy to just add an
> additional GET paramter to your browsers addressbar?
> Im not so familiar with the JavaScript XMLHttpRequest. Does it allow
> custom HTTP headers?

Adding HTTP headers to the XMLHttpRequest is indeed tricky. That's why 
we offer both options in the API.

I would recommend using curl for debugging instead of a browser. Just 
run this command in your *NIX shell of choice:

curl -k -v -X GET -H "Accept: application/json" -H "X-Version: 1.0" -H 
"X-Token: d8a8ab8893ea73f768b66b45234b5c3a" 
"https://api.flukso.net/sensor/c1411c6b4f9910bbbab09f145f8533b9?interval=hour&unit=watt"

>>
>> 2) I never liked the idea of using uuids to identify sensor as I always supposed this would mean that you would have to include all sensor uuids when you wanted to retrieve all sensor data for one controller; in that case I would have preferred to only transmit one uuid (the controller one); having uuids for the controller _and_ the sensors seem to give us the best of both worlds ->  I like that
> Ok, I can understand your concerns. Appending about 3*36 chars to your
> URL takes about 110 bytes. The maximum length of an URL isnt defined or
> limited by reference. Practically both browser and clientside supporting
> URLs with an total lenth of around 2KB. I think this is not an issue of
> practicability.
> It's more an issue of usability. Especially for us, the developers :p
>
> I also prefer the idea of grouping channels. But i dont know why to
> group them into physically based "controller-groups".
>
> We already have implemented a grouping feature which is indepedent from
> any physical conditions and allows you to group any kind of channel
> (tempsensor, watermeter, etc..) in virtual groups.
>
> A user can subscribe to these groups to easily get a "dashboard".
> A single channel isnt bound to a single user.
>
> A channel could be part of multiple groups.
> A group could be subscribed by multiple users.
> Groups could be nested.
>
> This sounds quite complex, i admit. But thats only our backend logic.
> Its quite flexible.
> I just introduced this schema to be save use of large changes in the
> future.
> In reality every user mostly just have one group, which is just
> subscribed by himself. This group contains for example three powermeters
> (one for each phase of an household).

We use a type and function field to categorize and identify the type of 
sensor we're dealing with.

> Later on, you may extend your installation. Possibly installing new
> meters to your vacation home or adding some temperature sensors.
> Now mixing these new meters with the existing ones is unhandy. Now you
> are able to to add these new channels to a new group.
> But thats only interessting for large installations. For example if you
> are an hirer and or hotel manager and have to controll a lot of
> channels.
>
> http://de.volkszaehler.wikia.com/wiki/Entwicklung/Daten%
> 26Referenzierung#Gruppierung
>
>> I'm not sure about the syntax; to me it seems best practice would be to have all parameters/values separated by slashes and not to use the "?" and "&" at all - am I wrong? Is this defined at all?
> I think thats part of the REST implementation:
> Every entity (channel, user, group, data) has its own url:
> http://localhost/users/9affbd90-9025-11df-9729-274054240631/channels?indicator=power
> http://localhost/users/9affbd90-9025-11df-9729-274054240631/groups
> http://localhost/channels/0e143bf0-9038-11df-a103-7b3971e2fb89/data?from=xxxxx&to=yyyyyyyy&groupBy=hour
>
> Adding entities should be done with an HTTP POST request.
> Editing entities should be done with an HTTP PULL request.
> Deleting entities with an HTTP DELETE request.
> If we are using an browser to access the backend we should pass this
> request method as an GET parameter:
> http://localhost/channels/0e143bf0-9038-11df-a103-7b3971e2fb89?action=delete
>
> Thats my interpretation of an RESTful API.

In a 'perfect' RESTful API (they almost never are), the URL (excluding 
the query params) defines a unique 'resource' which can be created, 
read, updated and deleted.

>
> Unfortunatly this requires some webserver configuration or mod_rewrite.
> According to our project name "VOLKSzaehler", our code should be easily
> to setup from everybody.
>
> mod_rewrite or extended webserver confiration isn't that easy and not
> allowed by every discount hoster..
>

We're using XEN VM's running Debian. Configure one of these VM's once, 
copy endlessly at will (testing, staging, live, another live, yal, ...).

>
> Now lets come to the offtopic part ;)
>
>>
>> BTW: did I mention the barcamp Mathias is planning for October 29th/30th in K'lautern?
>> http://developer.mysmartgrid.de/doku.php
>>
>> I'll be there; it would be nice if more vz members could arrange to go there as well
>
> Mhh, October i will starting my studies in Aachen. But I will do my very
> best to be there.
> @Bart: Justin told me that you live in Belgium? What do you think about
> to arrive together?

Sure, I'll pick you up in Aachen! Just buy me ein Helles in K'lautern. :-)

>
> I used the last weeks, and my vacation, to replace our old database
> abstraction layer and the object relation model with the new doctrine
> beta version. Its a quite interesting project with a lot of potential.
> Some key features:
> - dbal to a bunch of databases (sqlite, mysql, pgsql, oracle, odbc,
> mssql...)
> - automatic schema generation and update
> - object persistence
> - transactions
> - caching
>
> Have a look at http://www.doctrine-project.org/
>
> We now make also use of some nice PHP 5.3 feature: namespaces, late
> static binding and OOP of course.
> Our backend code is now "state of the art" ;)
>
> http://github.com/volkszaehler/volkszaehler.org/tree/doctrine
>
> wow what a novel...
>
> Regards Steffen
>

Cheers,
Bart.