Years ago, I built a chrome extension to import transactions into Mint. Mint hasn’t been updated in nearly a decade at this point, and once it stopped connecting to my bank for over two months I decided to call it quits and switch to LunchMoney which is improved frequently and has a lot of neat developer-focused features.

However, I had years of historical data in Mint and I didn’t want to lose it when I transitioned. Luckily, Mint allows you to export all of your transaction data as a CSV and LunchMoney has an API.

I’ve spent some time brushing up on my JavaScript knowledge in the past, and have since used Flow (a TypeScript competitor) in my work, but I’ve heard great things about TypeScript and wanted to see how it compared. Building a simple importer tool like this in TypeScript seems like a great learning project, especially since the official bindings for the API is written in TypeScript.

Deno vs Node

Deno looks cool. It uses V8, Rust, supports TypeScript natively, and seems to have an improved REPL experience.

I started playing around with it, but it is not backwards compatible with Node/npm packages which is a non-starter for me. It still looks pretty early in its development and adoption. I hope Deno matures and is more backwards compatible in the future!

Learning TypeScript

• You can’t run TypeScript directly via node (this is one of the big benefits of Deno). There are some workarounds, although they all add another layer of indirection, which is the primary downfall of the JavaScript ecosystem in my opinion.
  • ts-node looks like the easiest solution to run TypeScript without a compilation step. npm i ts-node will enable you to execute TypeScript directly using npx ts-node the_script.ts. However, if you use ESM you can’t use ts-node. This is a known issue, and although there’s a workaround it’s ugly and it feels easier just to have a watcher compile in the background and execute the raw JS.
• .d.ts within repos define types on top of raw JS. This reason this is done is to allow a single package to support both standard JavaScript and TypeScript: when you are using TypeScript the .js and .d.ts files are included in the TypeScript compilation process.
• Use npx tsc --init to setup an initial tsconfig.json. I turned off strict mode; it’s easier to learn a new typing system without hard mode enabled.
• Under the hood, typescript transpiles TypeScript into JavaScript. If you attempt to debug a TypeScript file with node inspect -r ts-node/register the code will look different and it’ll be challenging to copy/paste snippets to debug your application interactively.
• Same applies to debugging in a GUI like VS Code. You can enable sourcemaps, but the debugger is not smart enough to map variables dynamically for you when inputting strings into the console session. This is massive bummer for me: I’m a big fan of REPL-driven development. I can’t copy/paste snippets of code between my editor + REPL, it really slows me down.
• Similar to most other languages with gradual typing (python, ruby, etc), there are ‘community types’ for each package. TypeScript is very popular, so many/most packages includes types within the package itself. The typing packages need to be added to package.json. There’s a nice utility to do this automatically for you.
• If you want to be really fancy you can overload npm i and run typesync automatically.
• VS Code has great support for TypeScript: you can start a watcher process which emits errors directly into your VS Code status bar via cmd+shift+b.
• If you make any changes to tsconfig.json you’ll need to restart your watcher process.
• You can define a function signature that dynamically changes based on the input. For instance, if you have a configuration object, you can change the output of the function based on the structure of that object. Additionally, you can inline-assign an object to a type, which is a nice change from other languages (ruby, python).
  • Example of inline type assignment: {download: true} as papaparse.ParseConfig<Object>. In this case, Object is an argument into the ParseConfig type and changes the type of the resulting return value. Very neat!
• I ran into Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'Object'. No index signature with a parameter of type 'string' was found on type 'Object. The solution was typing a map/object/hash with theVariable: { [key: string]: any } . I couldn’t change the any type of the value without causing additional typing errors since the returning function was typed as a simple Object return.
• There’s a great, free, extensive book on TypeScript development.

One of the most interesting pieces of TypeScript is how fast it’s improving. Just take a look at the changelog. Even though JavaScript isn’t the most well-designed language, "One by one, they are fixing the issues, and now it is an excellent product." A language that has wide adoption will iterate it’s way to greatness. There’s a polish that only high throughput can bring to a product and it’s clear that after a very long time JavaScript is finally getting a high level of polish.

Linting with ESLint, Code Formatting with Prettier

• ESLint looks like the most popular JavaScript linting tool. It has lots of plugins and huge community support.
• You can integrate prettier with eslint, which looks like the most popular code formatting tool.
• VS code couldn’t run ESLint after setting it up. Had trouble loading /node_modules/espree/dist/espree.cjs. Restarting VS Code fixed the problem.

Here’s the VS Code settings.json that auto-fixed ESLint issues on save:

{
  "[typescript]": {
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {
      "source.fixAll.eslint": true
    },
  },

  "eslint.validate": ["javascript"]
}

And here’s the .eslintrc.json which allowed ESLint, prettier, and ESM to play well together:

{
    "env": {
        "browser": false,
        "es2020": true
    },
    "extends": [
        "standard",
        "plugin:prettier/recommended"
    ],
    "parser": "@typescript-eslint/parser",
    "parserOptions": {
        "ecmaVersion": 2020,
        "sourceType": "module"
    },
    "plugins": [
        "@typescript-eslint"
    ],
    "rules": {
    }
}

Module Loading

As with most things in JavaScript-land, the module definition ecosystem has a bunch of different community implementation/conventions. It’s challenging to determine what the latest-and-best way to handle module definitions is. This was a great overview and I’ve summarized my learnings below.

• require() == commonjs == CJS. You can spot modules in this format by module.exports in their package. This was originally designed for backend JavaScript code.
• AMD == Asynchronous Module Definition. You can spot packages in this style by define(['dep1', 'dep2'], function (dep1, dep2) { at the header of the index package. Designed for frontend components.
• UMD == Universal Module Definition. Designed to unify AMD + CJS so both backend and frontend code could import a package. The signature at the top of the UMD-packaged module is messy and basically checks for define, module.exports, etc.
• import == ESM == ES Modules. This is the latest-and-greatest module system officially baked into ES6. It has wide browser adoption at this point. This is most likely what you want to use.
• import requires module mode in TypeScript (or other compilers) not set to commonjs.
• If you use ESM, your transpiled JS code will look a lot less garbled, and you’ll still be able to use the VS Code debugger. The big win here is your import variable names will be consistent with your original source, which it makes it much easier to work with a REPL.
• There are certain compatibility issues between ESM and the rest of the older package types. I didn’t dig into this, but buyer beware.
• It looks like experimental support for loading modules from a URL exist. I hope this gets baked in to the runtime. There are downsides (major security risks), but it’s great for getting the initial version of something done. This was one of the features I thought was neat about Deno: you could write a script with a single JavaScript file without creating a mess of package*, tsconfig.json, etc files in a new folder.
  • https://unpkg.com is a great tool for loading a JS file from any repo on GitHub.
• You’ll get Cannot use import statement inside the Node.js REPL, alternatively use dynamic import if you try to import inside of a repl. This is a known limitation.. The workaround (when in es2020 mode) is to use await import("./out/util.js").
• When importing a commonjs formatted package, you’ll probably need to import specific exports via import {SpecificExport} from 'library'. However, if the thing you want to import is just the default export you’ll run into issues and probably need to modify the core library. Here’s an example commit which fixed the issue in the LunchMoney library
• When importing a local file, you need to specify the .js (not the ts) in the import statement import { readCSV, prettyPrintJSON } from "./util.js";

Package Management

• You can install a package directly from a GitHub reference npm i lunch-money/lunch-money-js
• You can’t put comments in package.json, which is terrible. Lots of situations where you want to document why you are importing a specific dependency, or a specific forked version of a dependency.
• npm install -g npm to update to the latest npm version.
• By default, npm update only updates packages to the latest minor semver. Use npx npm-check-updates -u && npm i to update all packages to the latest version. This is dangerous, and only makes sense if there are a small number of packages
• https://openbase.com is a great tool for helping decide which package to use.

JavaScript Learnings

• You’ll want to install underscore and use chain for data manipulation: _.chain(arr).map(...).uniq().value(). Lots of great tools you are missing from ruby or python.
• ES6 introduced computed property names so you can use a variable as an object key { [variableKey]: variableValue }
• I had trouble getting papaparse to read a local file without using a callback. I hate callbacks; here’s a promise wrapper that cleaned this up for me.
• Merge objects with _.extend.
• The dotenv package didn’t seem to parse .env with exports in the file. Got tripped up on this for a bit.
• require can be used to load a JSON file, not just a javascript file. Neat!
• There are nice iterators now! for(const i in list)
• There’s array destruction too const [a, b] = [1,2]
• Underscore JS has a nice memoize method. I hate the pattern of having a package-level variable for memoization. Just feels so ugly.
• There’s a in keyword that can be used with objects, but not arrays (at least in the way you’d expect).
• There’s a null-safe operator now. For instance, if you want to safely check a JSON blob for a field and set a default you can now do something like const accounts = json_blob?.accounts || []
• You are iterate over the keys and values of an object using for (const [key, value] of Object.entries(object))
• https://github.com/ccxt/ccxt is a neat project which transpiles JavaScript code into multiple languages.

Hacking & Debugging

• The most disappointing part of the node ecosystem is the REPL experience. There are some tools that (very) slightly improve it, but there’s nothing like iPython or Pry.
  • nbd is dead and hasn’t been updated in years
  • node-help is dead as well and just made it slightly easier to view documentaiton.
  • node-inspector is now included in node and basically enables you to use Chrome devtools
  • local-repl looks neat, but also hasn’t been updated in ~year.
  • The updated repl project wouldn’t load for me on v16.
• The debugging happy path seems to be using the GUI debugger.
• You can use toString() on a function to get the source code. Helpful alternative to show-source from ruby or ll from python. However, it has some gotchas:
  • It’s specifically discouraged since it’s been removed from the standard
  • Arguments and argument defaults are not specified
• It’s not obvious how to list local variables in the CLI debugger. There’s a seemingly undocumented exec .scope that you can run from the debugger context (but not from a repl!).
• You can change the target to ES6 to avoid some of the weird JS transpiling stuff,
• Run your script with node inspect and then before conting type breakOnUncaught to ensure that you can inspect any exceptions. I prefer terminal-based debugging, if you want to connect to a GUI (chrome or VS Code) use --inspect.
• There’s not a way I could find to add your own aliases to the debugging (i.e. c == continue == cont).
• It’s worth writing your own console.ts to generate a helpful repl environment to play with imports and some aliases defined. Unfortunately, this needs to be done on a per-project basis.
• You can’t redefine const variables in a repl, which makes it annoying to copy/paste code into a console.
  • It looks like there are some hacks you can use to strip out the const and replace with a let before the copy/pasted code gets eval’d. This seems like a terrible hack and should just be a native flag added to node.
• In more recent versions of node (at least 16 or greater), you can use await within a repl session.
• If you are in a debugger session await does not work, unlike when you are a standard node repl. You cannot resolve promises and therefore cannot interact with async code. This is a known bug, will not be changed, and makes debugging async code interactively extremely hard. Very surprised this is still a limitation.
• console.dir is the easiest way to inspect all properties of an object within a REPL. This uses util.inspect under the hood, so you don’t need to import this package and remember the function arguments.
• There’s a set of functions only available in the web console. Most of these seem to model after jQuery functions.

Open Questions

• How can I run commands without npx? Is there some shim I can add to my zsh config to conditionally load all npx-enabled bins when node_modules exists?
• Is there anything that can done to make the repl experience better? This is my biggest gripe with JavaScript development.
  • https://github.com/11ways/janeway looks interesting but seems dead (no commits in over a year)
  • This code looks like an interesting starting point to removing all const that are pasted into a repl.
• The number of configuration files you need to get started in a repo just to get started is insane (tsconfig.json, package*.json, .eslintc.json). Is there a better want to handle this? Some sort of single configuration file to rule them all?