Kind of puzzled about timestamps

Fri Mar 5 21:20:08 EST 2021

It would seem you have learned some of this the hard way.  And you are
indeed scaring me a bit.  Not sure what to do differently, however.  What
I'm hearing is properly dealing with time is hard and ugly.  I've
experienced some of this already.

I have been warned.  But, I still have to get on with things...  Can't
change the whole world, for that matter, I can't change how the cards are
written either.  So I'll have to put in (more than a few) tests to ensure
that I don't jam things up.

Thanks for passing on some of these pearls of wisdom.  I mean it.  When I
mess up, I'll remember, oh yeah, I was warned about that.  Next thought
will be, "Gosh, how am I going to fix this?"

On Fri, Mar 5, 2021, 8:54 PM Joshua Judson Rosen <rozzin at hackerposse.com>
wrote:

> On 3/5/21 2:15 PM, Bruce Labitt wrote:
> >
> > On 3/4/21 10:51 PM, Joshua Judson Rosen wrote:
> > >
> > > See also: "The Problem with Time and Timezones" <
> https://www.youtube.com/watch?v=-5wpm-gesOY>
> > >
> > > 😣
> >
> > That was somewhat comical.  Yeah, been trying to keep everything with
> > respect to UTC.  It can be a little difficult at times, as it's easy to
> > goof up and fall in to quite a few time trap holes.
>
> See also:
>
>         http://falsehoodsabouttime.com/
>
>
> http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html
>
>
> > One of the more difficult things has been indexing into the time array.
> > I've been using numpy's timedate64 and timedelta64 but occasionally
> > still get tripped up. Handling time is complicated.  Fortunately, all
> > that I care about for this project is relative time.  Start time, end
> > time and time is "linear" in between.  According to the the youtuber,
> > even that's not guaranteed if one spans the new year and we need a leap
> > second!
>
> Indeed! Though I fear that the reality is actually worse than the
> impression you got....
>
> A lot of the `if this happens and also' conditions are actually `if
> _either_ this _or_ that'.
>
> e.g.: most days have 86400 seconds, but...:
>
>         * some have 86401 (+ leap seconds)
>         * some have 86399 (- leap seconds--significantly rarer: hasn't
> happened _yet_, but...)
>         * some have 82800 (i.e. "some days only have 23 hours", normal
> spring-forward DST shift)
>         * some have 90000 (i.e. "some days have 25 hours", normal
> fall-backward DST shift)
>         * conceivably some may even have 90001 or 89999
>
> (Really! RE: negative leap seconds, `there is a first time for everything':
>  <
> https://www.livescience.com/earth-spinning-faster-negative-leap-second.html
> >)
>
> And yeah..., even if you're using unix time (seconds since the epoch)...,
> unix time
> specifically does _not_ count leap seconds..., which is both wonderful and
> terrible....
>
> Quoting the time(2) man page I have here:
>
>         This value is not the same as the actual number of seconds between
> the time and the Epoch,
>         because of leap seconds and because system clocks are not required
> to be synchronized
>         to a standard reference.  The intention is that the interpretation
> of seconds since the Epoch
>         values be consistent; see POSIX.1-2008 Rationale A.4.15 for
> further rationale.
>
> Wikipedia has some text on this, as well <
> https://en.wikipedia.org/wiki/Unix_time#Leap_seconds>:
>
>         When a leap second occurs, the UTC day is not exactly 86400
> seconds long and the Unix time number
>         (which always increases by exactly 86400 each day) experiences a
> discontinuity.
>         Leap seconds may be positive or negative. No negative leap second
> has ever been declared,
>         but if one were to be, then at the end of a day with a negative
> leap second,
>         the Unix time number would jump up by 1 to the start of the next
> day.
>         During a positive leap second at the end of a day, which occurs
> about every year and a half on average,
>         the Unix time number increases continuously into the next day
> during the leap second and then at the end
>         of the leap second jumps back by 1 (returning to the start of the
> next day).
>
>
> "all I have to care about is relative time" _should_ make your life
> easier..., in theory...,
> _assuming_ that the timestamps that you get and need to diff _really are_
> on a linear timescale.
>
> Good luck. I actually would love to hear about whatever linear timescale
> you end up settling on.
>
> This is why astronomers are using `Julian years'....
>
>
> Oh! ALSO: I think you may have mentioned previously that you're also
> reading these files
> from a FAT-formatted SD card or something..., which is, itself, multiple
> additional sources of confusion:
>
>         * FAT can only store timestamps down to *2-second* resolution,
> which means that
>           all file-timestamps get rounded to the nearest *even second*.
>
>         * FAT doesn't store timestamps on an _absolute_ timescale, it only
> stores them
>           (in `broken-down time') _relative to a given instantaneous
> timezone_.
>
>         * FAT doesn't actually give the timezone.
>
> Sooooo..., when you for example load a FAT-formatted SD card into a Linux
> computer,
> and the vfat driver in Linux needs to convert those `broken-down
> timestamps relative
> to an unspecified instantaneous timezone' into `absolute seconds since the
> epoch',
> IIRC it basically assumes that _Linux's current instantaneous system
> timezone offset_
> is appropriate for interpreting the broken-down time stamps in the FAT
> filesystem.
>
> If you are on the opposite side of a DST transition from when the files
> were stamped
> (even if it's only 1 second of difference... or, uh..., would that be 2
> seconds?);
> _or_ if you're actually in a different timezone (e.g. because you're in a
> different location),
> then the timestamps will convert incorrectly.
> em
> If you've ever noticed that the _filesystem_ timestamps _on_ the files in
> your digital camera
> are an hour (+/- 1 second...) off from the _EXIF_ timestamps _inside_ the
> files..., that's why.
>
> And the situation is both better and worse now with exFAT: the _precision_
> issue is basically fixed;
> the _accuracy_ issue of not knowing which timezone is _theoretically_
> fixed in that there is
> a timezone field in the timestamps..., but when dealing with embedded
> systems that write
> to exFAT you'll find that said timezone field is set to something
> arbitrary and bogus.
>
> --
> Connect with me on the GNU social network! <
> https://status.hackerposse.com/rozzin>
> Not on the network? Ask me for more info!
> _______________________________________________
> gnhlug-discuss mailing list
> gnhlug-discuss at mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.gnhlug.org/pipermail/gnhlug-discuss/attachments/20210305/40de7660/attachment-0001.html 