Semantics of timezone-aware datetime arithmetic

Wall time vs. absolute time semantics

Colloquially, time periods tend to be overloaded concepts, and what you mean when you say, e.g., "a day" or "a month" depends on the context. Looking at the following code, what would you expect the value for dt2 to be?

from datetime import datetime, timedelta
from dateutil import tz

NYC = tz.gettz('America/New_York')

dt1 = datetime(2018, 3, 10, 13, tzinfo=NYC)
dt2 = dt1 + timedelta(days=1)

There are two options, the first is using "wall time" semantics, returning the next day at the same time (rolling the clock forward by 24 hours): [3]

print(dt1)
print(wall_add(dt1, timedelta(days=1)))
# 2018-03-10 13:00:00-05:00
# 2018-03-11 13:00:00-04:00

The second option is to use "absolute time" semantics, where we jump forward to the point in time after which 24 hours have elapsed in the "real world". Because the start date here is immediately before a daylight saving time transition, these give different answers:

print(dt1)
print(absolute_add(dt1, timedelta(days=1)))
# 2018-03-10 13:00:00-05:00
# 2018-03-11 14:00:00-04:00

The concept of "add one day" is overloaded in this situation because the meaning of "1 day" can either mean "the period between two identical clock times on subsequent days" or it can mean "the period during which 24 hours have elapsed". I find that my intuition in this case is to use "wall time" semantics, because generally if I'm doing arithmetic on a non-UTC zone [2] it's because the time on the wall is what I care about; examples would include generating events that take place at the same time every week, scheduling a task to run during "off" hours, generating a bus schedule. In all those cases, you know what you want the "wall time" to look like and you can use the tzinfo object to determine where those times fit on the absolute timeline.

However, I find my intuition is a bit different when talking about subtraction:

dt1 = datetime(2018, 3, 10, 13, 30, tzinfo=NYC)
dt2 = datetime(2018, 3, 11, 8, 30, tzinfo=NYC)

print(wall_sub(dt2, dt1))
# 19:00:00

print(absolute_sub(dt2, dt1))
# 18:00:00

In this example, my intuition tells me that the result should be the number of hours that passed between those two points in time, because it's very rare for me to care about measuring the number of "clock hours" between two events like that, I'm almost always trying to figure out how much time has elapsed. If you naively coded up a datetime which has the "intuitive" behavior in each of those situations, you would find the bizarre result that the following relationship does not hold:

dt2 == dt1 + (dt2 - dt1)

Because the "intuitive" way to do things would be:

print(dt2 == wall_add(dt1, absolute_sub(dt1, dt2)))
# False

As far as I can tell, there is no way to satisfy everyone's intuitions in this case – if you choose absolute time semantics, you'll find that someone writes some variation on this code: [4]

DAY = timedelta(days=1)
dtstart = AbsoluteDateTime(2018, 3, 9, 12, tzinfo=NYC)
for i in range(4):
    print(dtstart + i * DAY)
    # 2018-03-09 12:00:00-05:00
    # 2018-03-10 12:00:00-05:00
    # 2018-03-11 13:00:00-04:00
    # 2018-03-12 13:00:00-04:00

and is confused why all of a sudden it started emitting dates at the wrong time! Similarly, if you choose wall time semantics, you'll find that someone writes some variation on this code:

dtstart = WallDateTime(2018, 4, 17, 12, tzinfo=NYC)
dtend = WallDateTime(2018, 4, 17, 12, tzinfo=tz.gettz('America/Los_Angeles'))

print(dtstart)
# 2018-04-17 12:00:00-04:00

print(dtend)
# 2018-04-17 12:00:00-07:00

print(dtend - dtstart)
# 0:00:00

and is confused about why the difference between these two datetimes – one 3 hours after the other – returns 0 hours!

Python's datetime semantics

Python uses fairly strange hybrid semantics for datetime addition and subtraction – it's probably not the compromise that I would choose, but there at least is a rhyme and a reason behind it.

For arithmetic within the same zone, all operations use wall time semantics. Considering that addition is always a "same zone" operation, since addition of a timedelta will produce a new datetime in the same zone, this gives intuitive behavior in the addition case, and subtraction uses wall time semantics because it is the inverse function to addition, so at least for the "same zone" case, dt2 == dt1 + (dt2 - dt1) is always satisfied.

Between zones, however, wall time arithmetic is basically meaningless; time zones can be considered the "units" of datetime calculations, so subtracting two datetimes in different zones is similar to subtracting X meters from Y feet – the answer is definitely not (X - Y), you need to convert the two quantities to the same units first. Python does this "unit conversion" by defaulting to "absolute time" semantics in the case of "between zone" calculations. You'll note that this difference gives rise to the same difference between "same" and "different" zone comparisons discussed in my previous post : since subtracting equal datetimes should always result in something equal to timedelta(0), equality semantics must depend on whether the operation is within a zone or between zones.

Unfortunately, because a timedelta does not carry time zone units of its own, the result of subtraction between two datetime objects is lossy. A timedelta can be generated from between-zone subtraction — in which case it represents an elapsed duration — but the resulting object is indistinguishable from an timedelta generated as the result of a "same zone" operation. Because of this, when between-zone subtraction crosses a DST boundary, it is no longer the inverse operation to addition [5]:

dt1 = datetime(2018, 3, 11, 1, tzinfo=NYC)
dt2 = datetime(2018, 3, 11, 1, tzinfo=tz.gettz('America/Los_Angeles'))

print(dt2 == dt1 + (dt2 - dt1))
# False

print(dt1 == dt2 + (dt1 - dt2))
# True

Conclusion

I am fairly convinced that there is no single intuitive approach to the datetime semantics problem – no matter what model a library or language chooses, you can't easily represent everything we mean when we do math on datetimes. I don't think Python does a perfect job with this, but as some of my example code indicates, it is possible to override Python's default semantics to get exactly the semantics you want; unfortunately, there are enough edge cases that just using the default semantics rarely leads to robust code. In a future post, I will cover some strategies on how to do robust datetime arithmetic in Python.

Footnotes