JSConfEU 2017 – Thanks Everyone!

Amazingly awesome crowd at JSConf EU today! Can’t believe how many people where here. Check it out:

//platform.twitter.com/widgets.js

Thanks everyone for listening to me the whole time.

A PDF version of the slides is here:

JS Library Design

This didn’t render well – so check back for fixes in the future.

Fixing JavaScript Date – Web Compatibility and Reality

In my previous post, I discussed things that could be fixed in JavaScript’s date implementation – if we wanted to. In this post, I’ll discuss things that can’t be fixed – no matter how much we want them to be.

No Semantic Versioning Here!

Most developers are familiar with Semantic versioning, or semver. In semver, we have the idea of three kinds of release:

  • Patch – releases that fix bugs only
  • Minor – releases that add small features
  • Major – releases that break things, so you have to change code

Semver is amazing. It clearly describes what versions we can safely upgrade to, when we should look for new features, and when we should expect to have to change code that is broken. It enables awesomeness. Unfortunately, it is not a reality for JavaScript.

A blessing and curse of JavaScript is that it is currently the world’s most popular programming language. Because of the sheer number of users, it’s safe to assume that anything that can be expressed with the language, has been expressed with the language, even if the code is what any sane person would consider terrible.

In addition, terrible code from 1998 is still being served to browsers everywhere, and there aren’t enough devs in the world to go update all of that code.

This results in two very important concepts that members of TC39 must constantly keep in mind:

  1. Web Compatibility – No change made to ECMAScript can be incompatible with the existing behavior of ECMAScript
  2. Web Reality – If code currently behaves a certain way, future versions of the spec should continue to have it behave that way – even if the behavior present is not described in the spec.

These concepts can really be summed up with the words “Don’t break the web!”. This creates a reality where there can be no such thing as a semver ‘major release’ in JavaScript. This idea drives every moment of the TC39 process, and has resulted in some unpopular but necessary compromises in the spec.

Mutability – a Web Compatibility Problem

I am a big fan of Domain Driven Design by Erik Evans. In the DDD world, objects can be differentiated as Entities which change over time and are tracked by their ID, and Value Types which are defined by their properties.  Under this definition, a DateTime is a value type. If any property of the date changes (for instance, the month changes from January to Feburary), the date is certainly a different date. Currently though, JavaScript doesn’t really work this way. Consider the following:

var a = new Date();
a.toISOString(); //"2017-04-05T05:57:53.350Z";
a.setMonth(11);
a.toISOString();//"2017-12-05T05:57:53.350Z";

As you can see, the value of object a changes. Yet, April and December are certainly different months, and these are certainly different dates. This kind of behavior sets people up for nasty bugs down the road. For instance, the following code will not behave as expected:

function addOneWeek(myDate) {
    myDate.setDate(myDate.getDate() + 7);
    return myDate;
}

var today = new Date();
var oneWeekFromNow = addOneWeek(today);

console.log(`today is ${today.toLocaleString()}, and one week from today will be ${oneWeekFromNow.toLocaleString()}`);
//today is 4/16/2017, 10:58:10 AM, and one week from today will be 4/16/2017, 10:58:10 AM

 

WOAH! This is no good. A better, and less bug prone behavior, would be to have the setters on the Date object return a new instance of the date – or for dates to be immutable. Then, the above code could be refactored to this common sense code:

function addOneWeek(myDate) {
    return myDate.setDate(myDate.getDate() + 7);
}
var today = new Date();
var oneWeekFromNow = addOneWeek(today);

console.log('today is ${today.toLocaleString()}, and one week from today will be ${oneWeekFromNow.toLocaleString()}');
//today is 4/09/2017, 10:58:10 AM, and one week from today will be 4/16/2017, 10:58:10 AM

Unfortunately, this is not to be because of a Web Compatibility issue. In short, if we were to make this change, tons and tons of code that relies on date being mutable (including the entire Moment.js library, BTW) would be broken.

Broken Parser – A Web Reality Issue

The ECMA262 standard currently describes very few rules for parsing date strings. These few excerpts are of particular interest:

ECMAScript defines a string interchange format for date-times based upon a simplification of the ISO 8601 Extended Format. The format is as follows: YYYY-MM-DDTHH:mm:ss.sssZ

This quote makes perfect sense. It states that JavaScript uses ISO8601 format as it’s main date interchange format. Since this is the most common interchange format for dates in modern computing, this is a great start! The standard then describes ISO8601 format options briefly:

This format includes date-only forms:

YYYY
YYYY-MM
YYYY-MM-DD

It also includes “date-time” forms that consist of one of the above date-only forms immediately followed by one of the following time forms with an optional time zone offset appended:

THH:mm
THH:mm:ss
THH:mm:ss.sss

So basically, you can have date only ISO8601 formats, and combined date-time formats. All good. But then a few lines further down, you get this wonderful quote:

When the time zone offset is absent, date-only forms are interpreted as a UTC time and date-time forms are interpreted as a local time.

In practice, what does this mean? Consider the following code:

new Date('2017-04-08').toISOString()
//"2017-04-08T00:00:00.000Z"
new Date('2017-04-08T08:30').toISOString() 
//"2017-04-08T15:30:00.000Z"

Basically, when I didn’t specify a time, the value was interpreted as a UTC value, but when I did specify a time it was interpreted as local. This is all… a bit mad. The next question one would ask is “is this some oddity of the ISO8601 specification?” But in fact that spec dictates that absent an offset, the time should be interpreted as local – meaning that under ISO8601 both values above should have been read as local time.

What happened here was a weird ‘Web Reality’.

In the days of ES5, the specification read this way:

The value of an absent time zone offset is “Z”.

This is saying that absent an offset, the time zone should be interpreted as UTC – the exact opposite of what the ISO8601 spec says. TC39 realized their error, and in ES2015 they corrected to this:

If the time zone offset is absent, the date-time is interpreted as a local time.

This is how the spec should read, as it aligns with ISO8601’s standard, and how basically every other date time API works. So why was it changed? Because as browsers started shipping this change, they started getting tons of bug reports that times weren’t being interpreted as they had been before – which of course they weren’t. By the time TC39 was able to give this issue more attention, the ecosystem was stratified, with some browsers picking UTC, and others picking Local, and others still making unique code compromises. After much evaluation of user feedback about what was expected to happen, the committee settled on the text as it is today, because it was the ‘reality’ of how the web worked – even if it wasn’t correct by the definition of the standard, or even particularly logical.

Given the amount of pain changes to this part of the spec caused the greater community, it is the case that it can’t be changed. The ‘web reality’ of what people expect from Date’s current parser will not allow it.

What do Do?

These two things were the impetuous for Matt, Brian, and I to choose to introduce a new datetime handling object to JavaScript, giving us a clean slate to make the world right. Currently we call this object ‘temporal’. This proposal can be found here, but look for a future post discussing our choices.

Fixing JavaScript Date – Getting Started

I’ve been off the blog for a while, which has to do with a lot of things going on in my life. That said, I’m happy to report that I’m back with stories about a big project – fixing the date handing in the JavaScript programming language itself!

History

A little backtracking here. I met Brian Terlson not too long after moving up to Redmond Washington to work for Microsoft. Brian is Microsoft’s representative to TC39, and the editor of the ECMA262 specification. 

I found Brian as part of a twitter conversation between me, Brendan Eich, and Moment.js co-maintainer, and fellow Microsoft employee Matt Johnson about how bad the Date handling in JavaScript is.

In this conversation, Brendan gave us some great history on the current Date object in JavaScript.

It is now common knowledge that in 1995 Brendan was given only 10 days to write the JavaScript language and get it into Netscape. Date handling is a fundamental part of almost all programming languages, and JavaScript had to have it. That said, it’s a complex problem domain and there was a short timeline. Brendan, under orders to “make it like Java” copied the date object from the existing, infant, java.Util.Date date implementation. This implementation was frankly terrible. In fact, basically all of it’s methods were deprecated and replaced in the Java 1.1 release in 1997. Yet we’re still living with this API 20 years later in the JavaScript programming language.

We all decided it was time for a fix and Matt, Brian, and I sat down to work.

Problems

In our first meeting, we identified the basic problems with the current Date implementation.
Here they are, in order of relative disgust:

  1. No support for time zones other than the user’s local time and UTC
  2. Parser behavior so unreliable it is unusable
  3. Date object is mutable
  4. DST behavior is unpredictable
  5. Computation APIs are unwieldy
  6. No support for non-Gregorian calendars

Fixable Stuff

Some of the issues mentioned are fixable in the current implementation of date. The lack of time zone support could be mitigated by adding a new factory function to date – something like:

var zonedDate = Date.inZone(zoneIdentifier, dateString);

This technique could probably also be used to support non-Gregorian calendars:

var calendarDate = Date.withCalendar('Hijri', dateString);

In that same vein, we could add APIs to Date to make computations easier. Right now, for instance, there is no way to add or subtract time – one instead has to perform a get/set behavior. To add one week to a date looks something like this:

var myDate = new Date();
myDate.setDate(myDate.getDate() + 7);

It wouldn’t be that big of a deal though, to spec something like an addDays() method, to make this a little nicer:

var myDate = new Date();
myDate.addDays(7);

The unpredictable DST behavior is another thing that can be fixed, and actually constitutes a bug in the current ECMA262 spec. This bug will be fixed in ECMAScript 2018 by this pull request. I’m planning on a later post to discuss the details of this change, and how “bugs” in the ECMA262 spec are handled.

Web Reality and Web Compatibility

Now we can see that lots of issues with date are fixable. However, we are left with two things on our list that aren’t: mutability and parsing. The reasons that these things can’t be fixed relates to two concepts that TC39 has to deal with a lot – web compatibility and web reality. These are the single most difficult things that TC39 has to reckon with, and I’ll tackle them in my next post.

An Aside: Standards committees are not a part of my regular job description as a SRE for Azure. This is a project driven from a passion for better JavaScript, and the OSS community. It wouldn’t be possible without the joint help of the JS Foundation, who I represent at TC39, and my division, Microsoft Site Reliability Engineering, which is allowing me to put work time and energy into improving technology. Thanks for believing I can do this, and giving me the resources to move forward.

Immutability – It’s Happening!

I’m super excited to highlight this awesome RFC from Lucas proposing to bring his Frozen Moment immutability plugin into the Moment.js core package as an optional add-on.

This RFC proposes a new Moment namespace, moment.frozen that wrappers all Moment functions with code that will automatically clone the Moment before performing mutations, creating an immutable API.

Moving forward, it is our desire to make the Frozen Moment immutable API the preferred way of interacting with Moment.js, effectively causing the library to become immutable. By making a second API, we allow legacy products to continue to function as before.

This all started when I wrote a blog post about why Moment isn’t immutable yet about a month ago. With the amount of enthusiasm we saw from the community, we decided to move forward with making Lucas’ plugin official.

As of right now, I am reworking our build process to allow us to better package this plugin.

In the meantime, we need community feedback on a few things:

  • Should this be packaged as a separate plugin, or just live in the core library as a second API?
  •  What should this API be named? Frozen is a working name that probably will not be adopted. We have discussed “m“ and “mom“ as possible options.
  •  Should the immutable API be a second global instead of being attached to the Moment global?
  • Should we simply make the “moment“ namespace immutable, and have a “momentLegacy“ namespace that users can use to overwrite the definition of moment for back-comparability if desired?
  • How should plugins and other libraries that depend on Moment interact with this API?
  • If we do overwrite the “moment“ namespace with an immutable one, should we release a compatibility build under a 2.0 version number? This build would have the same underlying code, but a mutable moment namespace.

Moving forward, we will be looking for contributors in a few key areas:

  • Documentation rework (needs to handle multiple APIs)
  • Package management
  • Build

In particular, someone with a UX background who wanted to help us improve the design of the docs page to support multiple APIs would be greatly appreciated.

Happy commenting!

Moment.js – Stuff we ARE doing!

Yesterday I wrote a post about how Moment.js isn’t immutable, and how it really isn’t in the cards for development right now.

I think as a team, we are sometimes not great about communicating things to our community that we are doing, and that are moving the library forward. As such, I wanted to write a short post about stuff we do have going on.

Next Major Release

The next major release of Moment.js and Moment TimeZone will have a complete rework of how the system handles time zones internally. This pull request is the completed work on the moment side. Tim has yet to get to the Moment TimeZone side, but it’s coming.

From a functionality standpoint for the library, this doesn’t really add much, though we might get in some syntactic niceties we didn’t have before. Internally though, we see a drastic improvement in our code base. This will allow us a lot more flexibility with new features down the road.

We also hope to have all of Moment changed over to Babel and Rollup by next major release. Again, this doesn’t add much for functionality, but it set us up to start using more ES2015 features. As a team, we have to decide what features we want to bring into the code and when. This is still in discussion.

There is no date for this, but all of this code is well in progress.

Other Stuff on the Table

If any of these are particularly important to you, let me know!