<div dir="auto">It would seem you have learned some of this the hard way. And you are indeed scaring me a bit. Not sure what to do differently, however. What I'm hearing is properly dealing with time is hard and ugly. I've experienced some of this already.<div dir="auto"><br></div><div dir="auto">I have been warned. But, I still have to get on with things... Can't change the whole world, for that matter, I can't change how the cards are written either. So I'll have to put in (more than a few) tests to ensure that I don't jam things up. </div><div dir="auto"><br></div><div dir="auto">Thanks for passing on some of these pearls of wisdom. I mean it. When I mess up, I'll remember, oh yeah, I was warned about that. Next thought will be, "Gosh, how am I going to fix this?"</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Mar 5, 2021, 8:54 PM Joshua Judson Rosen <<a href="mailto:rozzin@hackerposse.com">rozzin@hackerposse.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 3/5/21 2:15 PM, Bruce Labitt wrote:<br>
><br>
> On 3/4/21 10:51 PM, Joshua Judson Rosen wrote:<br>
> ><br>
> > See also: "The Problem with Time and Timezones" <<a href="https://www.youtube.com/watch?v=-5wpm-gesOY" rel="noreferrer noreferrer" target="_blank">https://www.youtube.com/watch?v=-5wpm-gesOY</a>><br>
> ><br>
> > 😣<br>
><br>
> That was somewhat comical. Yeah, been trying to keep everything with <br>
> respect to UTC. It can be a little difficult at times, as it's easy to <br>
> goof up and fall in to quite a few time trap holes.<br>
<br>
See also:<br>
<br>
<a href="http://falsehoodsabouttime.com/" rel="noreferrer noreferrer" target="_blank">http://falsehoodsabouttime.com/</a><br>
<br>
<a href="http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html" rel="noreferrer noreferrer" target="_blank">http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html</a><br>
<br>
<br>
> One of the more difficult things has been indexing into the time array. <br>
> I've been using numpy's timedate64 and timedelta64 but occasionally <br>
> still get tripped up. Handling time is complicated. Fortunately, all <br>
> that I care about for this project is relative time. Start time, end <br>
> time and time is "linear" in between. According to the the youtuber, <br>
> even that's not guaranteed if one spans the new year and we need a leap <br>
> second!<br>
<br>
Indeed! Though I fear that the reality is actually worse than the impression you got....<br>
<br>
A lot of the `if this happens and also' conditions are actually `if _either_ this _or_ that'.<br>
<br>
e.g.: most days have 86400 seconds, but...:<br>
<br>
* some have 86401 (+ leap seconds)<br>
* some have 86399 (- leap seconds--significantly rarer: hasn't happened _yet_, but...)<br>
* some have 82800 (i.e. "some days only have 23 hours", normal spring-forward DST shift)<br>
* some have 90000 (i.e. "some days have 25 hours", normal fall-backward DST shift)<br>
* conceivably some may even have 90001 or 89999<br>
<br>
(Really! RE: negative leap seconds, `there is a first time for everything':<br>
<<a href="https://www.livescience.com/earth-spinning-faster-negative-leap-second.html" rel="noreferrer noreferrer" target="_blank">https://www.livescience.com/earth-spinning-faster-negative-leap-second.html</a>>)<br>
<br>
And yeah..., even if you're using unix time (seconds since the epoch)..., unix time<br>
specifically does _not_ count leap seconds..., which is both wonderful and terrible....<br>
<br>
Quoting the time(2) man page I have here:<br>
<br>
This value is not the same as the actual number of seconds between the time and the Epoch,<br>
because of leap seconds and because system clocks are not required to be synchronized<br>
to a standard reference. The intention is that the interpretation of seconds since the Epoch<br>
values be consistent; see POSIX.1-2008 Rationale A.4.15 for further rationale.<br>
<br>
Wikipedia has some text on this, as well <<a href="https://en.wikipedia.org/wiki/Unix_time#Leap_seconds" rel="noreferrer noreferrer" target="_blank">https://en.wikipedia.org/wiki/Unix_time#Leap_seconds</a>>:<br>
<br>
When a leap second occurs, the UTC day is not exactly 86400 seconds long and the Unix time number<br>
(which always increases by exactly 86400 each day) experiences a discontinuity.<br>
Leap seconds may be positive or negative. No negative leap second has ever been declared,<br>
but if one were to be, then at the end of a day with a negative leap second,<br>
the Unix time number would jump up by 1 to the start of the next day.<br>
During a positive leap second at the end of a day, which occurs about every year and a half on average,<br>
the Unix time number increases continuously into the next day during the leap second and then at the end<br>
of the leap second jumps back by 1 (returning to the start of the next day).<br>
<br>
<br>
"all I have to care about is relative time" _should_ make your life easier..., in theory...,<br>
_assuming_ that the timestamps that you get and need to diff _really are_ on a linear timescale.<br>
<br>
Good luck. I actually would love to hear about whatever linear timescale you end up settling on.<br>
<br>
This is why astronomers are using `Julian years'....<br>
<br>
<br>
Oh! ALSO: I think you may have mentioned previously that you're also reading these files<br>
from a FAT-formatted SD card or something..., which is, itself, multiple additional sources of confusion:<br>
<br>
* FAT can only store timestamps down to *2-second* resolution, which means that<br>
all file-timestamps get rounded to the nearest *even second*.<br>
<br>
* FAT doesn't store timestamps on an _absolute_ timescale, it only stores them<br>
(in `broken-down time') _relative to a given instantaneous timezone_.<br>
<br>
* FAT doesn't actually give the timezone.<br>
<br>
Sooooo..., when you for example load a FAT-formatted SD card into a Linux computer,<br>
and the vfat driver in Linux needs to convert those `broken-down timestamps relative<br>
to an unspecified instantaneous timezone' into `absolute seconds since the epoch',<br>
IIRC it basically assumes that _Linux's current instantaneous system timezone offset_<br>
is appropriate for interpreting the broken-down time stamps in the FAT filesystem.<br>
<br>
If you are on the opposite side of a DST transition from when the files were stamped<br>
(even if it's only 1 second of difference... or, uh..., would that be 2 seconds?);<br>
_or_ if you're actually in a different timezone (e.g. because you're in a different location),<br>
then the timestamps will convert incorrectly.<br>
em<br>
If you've ever noticed that the _filesystem_ timestamps _on_ the files in your digital camera<br>
are an hour (+/- 1 second...) off from the _EXIF_ timestamps _inside_ the files..., that's why.<br>
<br>
And the situation is both better and worse now with exFAT: the _precision_ issue is basically fixed;<br>
the _accuracy_ issue of not knowing which timezone is _theoretically_ fixed in that there is<br>
a timezone field in the timestamps..., but when dealing with embedded systems that write<br>
to exFAT you'll find that said timezone field is set to something arbitrary and bogus.<br>
<br>
-- <br>
Connect with me on the GNU social network! <<a href="https://status.hackerposse.com/rozzin" rel="noreferrer noreferrer" target="_blank">https://status.hackerposse.com/rozzin</a>><br>
Not on the network? Ask me for more info!<br>
_______________________________________________<br>
gnhlug-discuss mailing list<br>
<a href="mailto:gnhlug-discuss@mail.gnhlug.org" target="_blank" rel="noreferrer">gnhlug-discuss@mail.gnhlug.org</a><br>
<a href="http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/" rel="noreferrer noreferrer" target="_blank">http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/</a><br>
</blockquote></div>