| Author |
|
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: January 05 2015 at 08:49 | IP Logged
|
|
|
Several times a day I get errors in my 1WIRE data, which leads to undesirable results.
At first I thought that just looking for a unexpected difference between the last data and the new data would be a way to detect these errors.
However, some of my data is taken far enough apart that a substantial difference can momentarily occur (ie, opening the freezer door can result in a 20 degree temp spike for a few minutes as warm air rushes in, but things are back to normal in minutes). I'd like to be able to note that temp spike.
So, I thought that looking for the "-2" error return from a bad data read, that PH puts in the data value, would be a good way.
But then I realized that the outside temps (here in Michigan) and the freezer temps can go thru "-2" so that doesn't work.
Dave how can one thus detect data error's?
The only way I can think of is to use the raw data passed instead of the cleaned up data. (ie, 203 vs 20.3)
Does the "-2" error occur in the raw data temp?
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: January 05 2015 at 17:46 | IP Logged
|
|
|
Ken,
I need more info. Where is the "-2" appearing? Does it show on the device status screen? Is in a local or temp variable as the result of a trigger? Is it the return from a function call?
The calc value is based upon the raw to calc formula which should be based upon the raw value. If you've left the raw to calc formula blank, then the default internal formula is always based upon the raw value is typically just a division by 10 or 100. For a temperature device, if the calc value is 72.4, then the raw value will be 724. If you compare the raw to the calc and you get something "out of whack", then I would suspect the raw to calc formula having an issue.
Dave.
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: January 05 2015 at 18:05 | IP Logged
|
|
|
In the Analog I/O Device definitions the 'raw to calc' is blank.
On the Trigger the Boolean formula (trying to skip bogus readings) is set to ...
Code:
if(abs(number(ph_getanalogcalcstat("1WIRE-OUT")) - ph_getvar_n(2,5))>15,0,1)
/* If there is more than a 15 deg difference assume this is a bad read and ignore it, leaving previous value in place. */ |
|
|
The Action formula on the Trigger is (trying to average the old/new values [before the new averaging capabilities in 215c])...
Code:
if(abs(number(ph_getanalogcalcstat("1WIRE-OUT")) -
and the Trigger Action field is...
[code]ph_setanalogvalue("1WIRE-OUT",(ph_getglobal_n("TEMP-OUT") + ph_getvar_n(2,5))/2*10, string((ph_getglobal_n("TEMP-OUT") + ph_getvar_n(2,5))/2,"0.0"),"°F",31) +
ph_setglobal_s("TEMP-OUT",ph_getanalogcalcstat("1WIRE-OUT")) |
|
|
This results in the {TEMP-OUT} global getting set to -2 when an error occurs, and this -2 gets charted as a temperature.
Should I be using something other than TEMP5 for my temperature value in order to catch the -2 as an error, rather than a temperature?
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: January 05 2015 at 19:45 | IP Logged
|
|
|
Ken,
Checked out the 1-wire controller code and nowhere is there code to return an error or a -2 value. Basically it reads the 1-wire controller and whatever it receives is what it passes on. The first thing I'd like to know is what are the actual values returned from the 1-wire controller when the global gets set to -2. This should be in the eventlog and should show the raw value and what it changed from.
Next, the 1-wire controller passes the old rawvalue, the new rawvalue, and the new calcvalue to the trigger so no need to do the ph_getanalogcalcstat in the boolean field. One of the TEMP vars will contain the old value. Also, since you've got this formula on the boolean that doesnt allow the trigger to fire (and change the global to -2) when the difference is > 15, then I wouldnt think the incoming calc value is bad (its at least within 15 of the old value).
Since the global {TEMP-OUT} is being set to -2, lets look at the part of the formula that assigns the calc value (since this is where the global is set from). You've got:
Code:
| string((ph_getglobal_n("TEMP-OUT") + ph_getvar_n(2,5))/2,"0.0") |
|
|
I would check the eventlog and see what the value of TEMP-OUT global is before it got changed to -2. Im not sure what is a normal value for this. The thing that leads me to believe that this is some kind of formula error is that you say the error is always a - 2. If it was a bad value coming from the 1-wire controller, then it would be dependent upon the current value in TEMP-OUT which I suspect is constantly changing.
What is most like happening is not a bad 1-wire data but something wrong in the formula that is ultimately resulting in a -2 and this is what gets set to the calc value and ultimately the TEMP-OUT var. It would be interesting to see what the raw value is in this situation.
There are a few things I would change with your ph_setanalogvalue function. The first thing I would is subtract a 16 from your flags parm (set it to 15 instead of 31). The 16 forces a trigger to fire...most likely the same trigger that is in the process of firing. This would cause an endless loop situation. We're lucky this time around though because the 1-wire analog types DO NOT currently fire the trigger even if specified (Ive added this to my to-do list so it will soon cause you alot of headaches). The other thing I would probably do is work solely with the raw value, place the desired formula in the raw to calc field and then have the ph_setanalogvalue function perform the rawtocalc calculation by just passing an empty string for the calc parm and adding 64 to flags parm.
Anyways, I think we'll be able to dig deeper once I see the eventlog entries for when the 1-wire device changed and when the global changed.
Dave.
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: January 06 2015 at 15:47 | IP Logged
|
|
|
I'm beginning to suspect that I may have had a "senior moment".
I have made so many PH changes the last few weeks and have been monitoring so many unexpected behaviors that I fear I may be mixing apples and oranges.
The 1WIRE incidents, I think, were a result of the freezer actually getting to -2° (not an error value) while at the same time a 1WIRE Outside temp value was missed and thus went to zero, making it look like the two were the same and clicking on the time/temp value on my graph showed "-2", but that mouse-click response is always for the foreground chart line which must have been the freezer, not the outside temp. 
On further examination, it appears that my "-2" error issues are really coming from the Thermostat (STAT) Analog device support, so that is where I will be focusing over the next few days.
PS - the need for correcting calc's has virtually been eliminated by the new averaging capability in PH 215c. Waiting to see how that works, but should know in 24 hours!
Thanks for that new capability Dave. 
[:-)}
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: January 06 2015 at 21:47 | IP Logged
|
|
|
Ken,
Glad to hear it. I traced every function in your formulas last night and could not find a one that would return a -2 as an error value (some functions do return values such as a negative number but none of these do). I was very perplexed in how it could happen so glad to hear that its nothing in the code.
Also good to hear that the new averaging capability is working for you. Ive decided to use it myself and have been able to reduce my temperature "deadbands" from .5 degrees to .2 degrees (Im only using 3 samples though so things can still change pretty quick).
Dave.
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: January 07 2015 at 06:38 | IP Logged
|
|
|
Question on averaging, Dave.
From your comment above, it sounds like a Trigger is fired AFTER the data average is formed.
Since I have a lot of jitter on some of my 1WIRE sensors (especially outside where wind gusts can quickly change the reading for a moment before it bounces right back to where it was) it sounds like averaging will greatly reduce the jitter triggers. To minimize constant triggering, I've had to crank the OUTSIDE deadband to over 10 (1 degree). I'd love to get it to 5 (0.5 degree).
Guess I'll try, this morning and report back what happens. 
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: January 07 2015 at 09:32 | IP Logged
|
|
|
Maybe its doing what I hoped, but I'm not sure. I have the deadband for the 1WIRE outside sensor set to 6 (0.6° change) and the averaging set to 103 (3 samples) yet I an getting more triggers than I expected.
It is surprising to me that all within the same time slice (10:12:59) that three trigger events would occur. I would not have expected the OUT sensor to slide so quickly thru a change range, especially if a 3 sample average was imposed on it.
Do I just need to average over a longer sample, or increase the deadband? I was hoping to get a ±0.5° resolution for the temp.
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: January 07 2015 at 13:10 | IP Logged
|
|
|
Ken,
You are correct. The averaging is done before a trigger gets to fire. When 1-wire data is collected, there is basically two values being stored. The first one is the average data and the second one is the raw data that is in the Analog IO table.
When a 1-wire reading comes in, if averaging is enabled, the raw reading is averaged into the average value. This average value is then compared to the raw value in the analogio table and if it exceeds the "deadband", then the average value is copied to the raw analogio value.
When the raw analogio value is changed, we then will get a raw to calc operation performed and a trigger check. The trigger for a 1-wire device ALSO has a "deadband". Setting this "deadband" to 0 will mean the trigger will fire everytime the raw value changes. But..you can set the "deadband" for the trigger higher than the "deadband" for the raw value and this will decrease your triggers while still allowing you higher resolution for display.
Looking at your log though, I suspect a problem with your 1-wire sensor. You're getting some fairly large swings within a very limited timespace which is not something Ive seen with any of my sensors. I definitely get the occasional reading that seems out of whack and the readings that seem to toggle back and forth between two values and the averaging works very well with this but yours appear to be something different.
Now we don't have access to the actual values that came in and only see the averaged values that exceeded the deadband so the actual values would be even more exaggerated than what we see here.
Judging from your log, if the global updates shown are tied to the 1-wire updates shown (they should be based upon time), then you're still doing some kind of averaging or other as the globals arent tracking to the 1-wire values.
So a simple answer to your question is that increasing the sample size will smooth the value even more. Im surprised that your 1-wire controller can sample that fast. Ive got a total of 7 1-wire devices and my refresh rate is about once a second. Depending upon how fast you'd like to respond to temperature changes and how fast your actual poll rate is would probably determine your average sample count. If you want to see a change at least once every 10 seconds and you're able to poll the devices 3 times per second, then I would try an average sample count of 30.
Dave.
|
| Back to Top |
|
| |
GadgetGuy
Super User
Joined: June 01 2008
Location: United States
Online Status: Offline
Posts: 942
|
| Posted: January 07 2015 at 14:50 | IP Logged
|
|
|
The Global changes are indeed tied to the triggers. These are the values I am using everywhere else, so I really want to get them right.
My 1wire OUT device is defined as (there is no Raw to Calc field entry)....
And my Trigger is...
I see that the Global is indeed changing with a highly deadbanded and averaged value.
Increasing the sample size didn't seem to reduce the Trigger trigger, only the Global changes.
When you say "set the "deadband" for the trigger higher than the "deadband" for the raw value and this will decrease your triggers while still allowing you higher resolution for display." did you mean to change the Boolean test in the Triggers entry for this 1wire device?
Also when you say...
If you want to see a change at least once every 10 seconds and you're able to poll the devices 3 times per second, then I would try an average sample count of 30.
I am a bit puzzled as I didn't think the potential "sample" rate had anything to do with data collection. I thought the data updates only came when the 1wire device exceeded the deadband thresholds. I don't care about timed data capture. I just want to know when the sensor is reporting more than ±0.5 degrees of change (so I can update my Global value).
If the averaging is only done when the data value changes, I'm concerned an averaging count of 30 (130 in the Point field) would result in highly unpredictable readings. If there is jitter, things will be updated, but in the absence of jitter, couldn't it be hours before enough normal data would be gathered and thus just "flat-lined" so to speak?
__________________
Ken B - Live every day like it's your last. Eventually, you'll get it right!
|
| Back to Top |
|
| |
dhoward
Admin Group
Joined: June 29 2001
Location: United States
Online Status: Offline
Posts: 4447
|
| Posted: January 07 2015 at 16:16 | IP Logged
|
|
|
Ken,
The 1-wire controller "polls" the 1-wire devices as fast as it can in an endless loop. How often a device gets polled is a factor of usually how many devices (and device types) you have. You seem to be getting your outdoor temp sensor polled at least 3 times a second. My devices get polled about once a second. If someone had 15 devices on their 1-wire network, each device may get updated once every two seconds.
So a 1-wire device is polled endlessly as fast as the controller and number of devices allow. This polled reading is then figured into the running average if you have averaging enabled. If you are getting 3 reads / second and have an average sample count of 30, then your oldest polled reading will factor out after 10 seconds.
Every time a reading is figured into the running average (in your case it looks like you're getting 3 readings / second), a comparison is made between the running average and the current stored raw analog io value (which itself is a snapshot of the running average when the deadband was last exceeded). If this comparison exceeds the deadband, the current running average is copied to the raw analog io value (becoming the new raw value) and the raw to calc formula is applied to create a new calc value. These values are all then applied to the trigger check routine.
I hope this makes sense as I don't know how else to explain it.
The trigger deadband for a 1-wire device is the "Trigger ID Number" column. This is how much the raw value has to vary in order for the trigger to fire.
On your 1-wire analog I/O definition, you've got a point value of 102 which means you're only averaging 2 samples. I would increase this to 130.
I cant see the full action for your 1WIRE-OUT trigger. Just that you're setting the global to probably the calc value but based upon your log entries I see the 1-wire change from 169 to 163 and the corresponding (I assume) global var change entry shows the value changed from 16.9 to 15.0 which is not right. It should have changed from 16.9 to 16.3 which leads me to believe that you're doing some kind of manipulation of the raw value before you assign it to the global which is not what I think you want.
Dave.
Edited by dhoward - January 07 2015 at 16:19
|
| Back to Top |
|
| |
|
|
Active Topics 
Memberlist 
Search 
Help
Register 
Login
PowerHome Messageboard : PowerHome Programming
Topic: Detecting Errors in 1WIRE data
