| Author |
|
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 07 2022 at 17:30 | IP Logged
|
|
|
Dave (Oh Wondrous & Wise one)-
You may recall me asking for help to "scrape" the web page of my TempStick ()https://tempstick.com/ after they changed from a "normal" HTMP web page to a totally API driven set of links.... making it impossible to get any embedded Temp/Humidity data.
After many months of dialog with them they finally revealed a beta version of an API approach to access the desired data.
Unfortunately their API approach is totally unfamiliar to me, but I am hoping that in your near infinite wisdom you might be able to provide a solution that will allow ph to grab the data.
I can do it in two lines of code but that is with the help of an API Builder App called "Postman"
Using that to enter the API Key assigned to me and a single curl GET command I can get the TempStick data.
Code:
"type": "success",
"message": "get sensors",
"data": {
"groups": [],
"items": [
{
"id": "38285",
"vers ion": "1102",
"sens or_id": "2766956",
"sens or_name ": "2766956",
"sens or_mac_ addr": "3C:71:BF:2A:38:6C",
"owne r_id": "24526",
"type ": "DHT",
"aler t_inter val": "1800",
"send _interv al": "1800",
"last _temp": 33,
"last _humidi ty": 75.12,
"last _voltag e": "2.84",
"wifi _connec t_time": "2",
"rssi ": "-56",
"last _checki n": "2022-07-07 21:45:15",
"next _checki n": "2022-07-07 22:15:15",
"ssid ": "burkLAN-2G",
"offl ine": "0",
"grou p": 0,
"use_ sensor_ settings": "0",
"use_ offset" : "0",
"temp _offset ": "0",
"humi dity_of fset": "0",
"use_ alert_i nterval": "0",
"aler t_temp_ below": "",
"aler t_temp_ above": "",
"aler t_humid ity_below": "",
"aler t_humid ity_above": ""
}
]
}
} |
|
|
The "last_temp" and "last_humidity" parameters is all I need.
Unfortunately I can find no way to get that Response from Postman and displayed on the screen into a file or clipboard for ph to work with. Worse I have not been able to even discover how to execute the cURL commands. I've tried HTML web pages, windows command prompts, batch files but with no real success. Biggest issue is how to input the API Key, followed by a GET command. Everything I have tried sees them as two separate commands and not as a pair, so I can't obtain Authorization.
According to TempStick's support team (who don't know much about this API) things are built around REST which is just HTTP / CURL requests. So most of the queries are just a "GET" HTTP request with an HTTP header for the API Key. Also, that Postman is software you can download that makes working with the API easy and it can produce code snippets in different programming languages
FYI.... here is the limited TempStick documentation on their API >> "https://tempstickapi.com/docs/"
My X-API-KEY: fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06
The following command will retrieve the stick data for scraping (as listed way above)...
GET https://tempstickapi.com/api/v1/sensors/all
If you can just get the Authorization API Key into the request header and then extract the Response data for ph use that's all it will take.
Is this in the realm of possibility? If not can you point me to a starting point. Where do you put those commands (ie, what application holds them and how do you send them)? I had a little success with the Windows Command prompt approach but couldn't combine Authorization with the GET.
P.S. - Just got this code snippet from Postman that supposedly will do the trick, but I can't make it work..
Code:
curl --location --request GET 'https://tempstickapi.com/api/v1/sensor/2766956/readings' \
--header 'X-API-KEY: fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06' |
|
|
Holler if there is any more info I need to send you.
gadgetGuy
Edited by GadgetGuy - July 07 2022 at 17:44
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 07 2022 at 17:55 | IP Logged
|
|
|
Here is a supposed JavaScript code snippet, but I'm not sure how to utilize it so couldn't test.
Code:
var myHeaders = new Headers();
myHeaders.append("X-API-KEY", "fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06");
var requestOptions = {
method: 'GET',
headers: myHeaders,
redirect: 'follow'
};
fetch("https://tempstickapi.com/api/v1/sensor/2766956/readin gs", requestOptions)
.then(response => response.text())
.then(result => console.log(result))
.catch(error => console.log('error', error)); |
|
|
gadgetGuy
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: July 07 2022 at 19:24 | IP Logged
|
|
|
Hi Ken,
The below formula will extract both the temperature followed by humidity separated by a space:
Code:
ph_rtne(ph_setvar_s(1,1,ph_geturl1("http://tempstickapi.com/ api/v1/sensors/all","X-API-KEY: fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06",5,5000)) +
ph_setvar_s(1,1,ph_flattenjson(mid(ph_getvar_s(1,1),pos(ph_g etvar_s(1,1),"~r~n~r~n") + 4),":",0,1))) + ph_extractval(ph_getvar_s(1,1),"data.items.0.last_temp",":") + "
" + ph_extractval(ph_getvar_s(1,1),"data.items.0.last_humidity", ":") |
|
|
There are several ways you can do this but the key part is the: ph_geturl1("http://tempstickapi.com/api/v1/sensors/all","X-A PI-KEY:
fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06",5,5000)
I couldnt get it to work with https: but that could be because I don't have the latest TLS libraries on my PowerHome machine. ph_geturl1 type 5 was the only one that
would return the JSON data to me. I basically assign the JSON to [LOCAL1], chop off just the JSON in [LOCAL1] and then flatten the JSON in [LOCAL1]. With the flattened
JSON, I then use ph_extractval to extract the temperature and humidity. You can also do this without flattening the JSON and just using a regular expression to extract
the data like below:
Code:
ph_regexsnap3(',"last_temp":([^,]*),"last_humidity":([^,]*),','Temperature: $1, Humidity: $2', ph_geturl1("http://tempstickapi.com/api/v1/sensors/all","X-A PI-KEY:
fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06",5,5000), 1,0,1,0,0) |
|
|
It's shorter but a benefit is that you can extract the data and format the output string all at once.
Hope this helps,
Dave.
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 07 2022 at 19:45 | IP Logged
|
|
|
Wow!! Pure genius!!!
Complex enough that I couldn't follow what's gong on. It
will probably take some time, and breaking things down
into pieces to see what each part is doing before I fully
grasp the full process.
It's not clear to me how the 2nd code snippet fits into
the 1st one. Exactly what would #2 replace in #1?
I'll need to put the temp in one Local and the humidity
in another as they are separate values that get entered
into a graph table, and I should then have what I need.
Many, many, many, many THANKS for your effort. I've been
horsing around with it for over 3 days and making very
little actual progress.
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: July 07 2022 at 20:03 | IP Logged
|
|
|
Ken,
Glad to help. Code snippet 2 entirely replaces code snippet 1 (it's an alternative).
I didnt actually test it, but the following code should put the temperature in [LOCAL2] and the humidity in [LOCAL3]:
Code:
ph_setvar_s(1,1, ph_geturl1("http://tempstickapi.com/api/v1/sensors/all", "X-API-KEY:
fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06",5,5000)) +
ph_setvar_s(1,1,ph_flattenjson(mid(ph_getvar_s(1,1), pos(ph_getvar_s(1,1),"~r~n~r~n") + 4),":",0,1)) +
ph_setvar_s(1,2,ph_extractval(ph_getvar_s(1,1), "data.items.0.last_temp",":")) +
ph_setvar_s(1,3,ph_extractval(ph_getvar_s(1,1), "data.items.0.last_humidity",":")) |
|
|
Modifying code snippet 1 seemed to make the most sense for this application.
Dave.
Edited by dhoward - July 07 2022 at 20:06
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 08 2022 at 17:24 | IP Logged
|
|
|
Dave-
When I started breakiing down your last snippet I couldn't get it to work in even the simplest form.
I changed the LOCAL save to a GLOBAL save so I could easily reference the data in subsequent macros.
Using the following code...
I got the same error I always got when trying to access the data with curl commands....
Were you able to actually run your snippet and get results? If so have I constructed something incorrect?
ken
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: July 08 2022 at 18:32 | IP Logged
|
|
|
Ken,
The last formula I posted, I didnt actually run last night. I did take it just now though, pasted into the PowerHome Multi-Editor, removed the CR/LF's that the
forum inserts and made sure that the header was properly formatted and it returned just fine.
I see from your screenshots, you're running the ph_geturl1 function and your return is an error saying that the X-API-KEY is a required header. That tells me
that the formula you ran has not proper for the header parameter and is probably either an embedded CR/LF and/or a missing space. The formula should all fit on a
single line (no CR/LF) and the header field must appear exactly like this:
"X-API-KEY: fc7956bbfe03055cf1d8bcd4e5a0dcb64bc2701b25578a5b06"
In case the forum goofs it up, it should be X-API-KEY with no spaces, a colon, a space (this is important), followed by your API key with no spaces.
Dave.
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 08 2022 at 19:59 | IP Logged
|
|
|
Dave-
You caught it!!!! I just copied your post above, pasted it into NotePad found a CR/LF and removed it so it looks like this now....
That is the Formula now pasted in the ph Macro Formula field of a Set Global command, which when run sets the TSDATA-RAW Global to exactly what I need. I can Regex strip the parameters out of that now.
A MILLION thanks. Without your insights I would have worked on this for weeks with no results.
Your generosity and caring is extraordinary, especially as busy as I know you are.
You ARE DA MAN!!!!!
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 14 2022 at 19:09 | IP Logged
|
|
|
Dave-
I just added a 2nd sensor to my environment monitoring collection and as a result the data returned by the API GET effort increased from 690 characters to 1309 which now results in failure.
Apparently the ph_geturl1() function can only return 1024 characters and anything beyond is truncated.
Thus I can't capture the entire data stream from the 2 sensors that report in as a pair.
Is there a way to get the entire data string?
Also did you see my new email function fail post (in the Programing Section).
gadget
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 15 2022 at 07:33 | IP Logged
|
|
|
Dave-
Ignore my previous 1024 char question. I think it makes most sense to just use two GETs to extract each sensor's data independently.
Accordingly I will need to use the following "GET Sensor" API call...
https://tempstickapi.com/api/v1/sensor/:sensor_id
So far I can't get it to work, but I think the issue is in the...
ph_flattenjson(mid(ph_getvar_s(1,1), pos(ph_getvar_s(1,1),"~r~n~r~n")
function, which is what is failing.
I can't figure out what the "~r~n~r~n" search string is for as there are not 2 blank lines (~r~n~r~n) anywhere in the RAW data string.
Unfortunately once I added the 2nd TempStick I can't undo it as it is permanently stored on the TempStick Server somewhere in the world, and that 2nd instance (for now has broken our code so I can't get a copy of what the RAW data looked like before). So making debugging a bit difficult, but I'll keep at it.
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: July 15 2022 at 10:27 | IP Logged
|
|
|
Got it !!
Browsing the Forum (ya gotta love this place) I discovered
that [LOCALs] and [TEMPs] are not db items but stored in
memory and have unlimited length.
Used them to handle my long strings and checked by writing
each to a file so I could read the entire data string.
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: July 16 2022 at 00:33 | IP Logged
|
|
|
Hi Ken,
Glad I was a little slow getting to this because I would have been spinning my gears. Looks like you figured out that the ph_geturl1 function isnt
actually limited to 1024 characters and it's actually a global variable (stored in the database) that is limited to 1024 characters (this is what
I would have missed if not for your later emails and I would have been down a rabbit hole trying to figure out what was wrong with the ph_geturl1
function).
Locals, Temps, and Global SYSTEM variables are all stored in memory and can be virtually any size. It seems very few people use the global system
variables and that is probably due to confusion with global database variables. Even though the help file is wrong on some of the functions saying
there are only 20 Global system variables, there is actually 100 global system variables. As the name implies, these variables are global and the
values will persist until overwritten or you shut PowerHome down (local variables and to some extent temp variables are reset when a new macro or
formula is run).
Anyways, the last question I saw was about the "~r~n~r~n" in the search string. This is because some of the ph_geturl1 types return the HTML
header information (at the very beginning) which is then followed by the double cr/lf before returning the actual HTML data. If I use the
ph_geturl1 function with a retrieve type of 5, I get something like this returned:
Formula Evaluation
Execution time: 0.655 seconds.
The formula evaluates to: 200
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> ;
<html xmlns="http://www.w3.org/1999/xhtml" >
<head><title>
As you can see, this particular type only returns the HTML return code (instead of the full headers) which is "200" followed by double cr/lf
followed by the actual HTML.
Hope this helps,
Dave.
|
| Back to Top |
|
| |
|
|
Active Topics 
Memberlist 
Search 
Help
Register 
Login
PowerHome Messageboard : PowerHome General
Topic: Extracting TempStick weather Parameters
