Fixing Invalid DATABASE_URL Reference on CircleCI

I've become a huge fan of CircleCI—it's the best CI tool for small-medium projects that I've found. Recently I ran into an issue setting up a new rails project with Postgres as the database backend with circle:

export RAILS_ENV="test" export RACK_ENV="test" bundle exec rake db:create db:schema:load --trace ** Invoke db:create (first_time) ** Invoke db:load_config (first_time) ** Execute db:load_config rake aborted! NameError: Cannot load `Rails.application.database_configuration`: uninitialized constant DATABASE_URL

However, when I looked at my database.yml the constant DATABASE_URL was clearly not being used. DATABASE_URL was properly being used as the lookup string for an ENV reference:

# config/database.yml test: url: <%= ENV['TEST_DATABASE_URL'] %>

I was stumped for a while until I ssh'd into circle and noticed that somehow the single quotes were stripped from DATABASE_URL:

# cat config/database.yml test: url: <%= ENV[DATABASE_URL] %> adapter: postgresql database: circle_ruby_test username: ubuntu host: localhost

Circle's build inference process involves generating a database.yml file which uses the original database.yml file to some extent. Some part of the process involves stripping single quotes, which is what was causing the build breakage. Make sure that you use double quotes instead of single quotes in database.yml when using CircleCI.

Automating Staging Deploys to Staging on CI Build Success

Here's how to automatically push new builds (and run database migrations) to a heroku-hosted app using circle CI:

deployment: staging: branch: master commands: - heroku maintenance:on --app the-app-name - git push $CIRCLE_SHA1:master - heroku run rake db:migrate --app the-app-name - heroku maintenance:off --app the-app-name

Continue Reading

Sending Dokku Container Logs to Papertrail

I'm a huge fan of Heroku. Although it's expensive compared to a raw DigitalOcean droplet or EC2 instance, it's so easy and it just works.

However, there are some use-cases that Heroku doesn't handle well. Business integration systems (eCommerce NetSuite integration, for example) often require handling requests that may take longer than the 30 second request timeout.

Dokku is the next-best alternative to using heroku (if you don't have enormous scaling needs: dokku apps are restricted to a single host). Here's how to push your dokku app logs running to PaperTrail, or another syslog-compatible cloud hosting provider (I'm sure this approach would work for Loggly or CloudWatch).

Install dokku-logspout: dokku plugin:install Point logspout to your PaperTrail syslog endpoint: dokku logspout:server syslog+tls:// Make sure your PaperTrail endpoint is configured to accept TLS connections. Install dokku-hostname: dokku plugin:install dokku-hostname. This will ensure that the host in your PaperTrail logs use the heroku-like name of your worker instead of the docker container ID. Run dokku logspout:start and use logspout:stream to ensure that your logs are being aggregated properly.

The resulting log output should look something like:

Apr 15 14:49:28 pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:167:in `stats' Apr 15 14:49:28 pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:127:in `process' Apr 15 14:49:28 pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:79:in `process_one' Apr 15 14:49:28 pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:67:in `run' Apr 15 14:49:28 pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:16:in `watchdog' Apr 15 14:49:28 pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:24:in `block in safe_thread' Apr 15 14:49:28 pipeline.worker.1: 11 TID-os97lykxo INFO: [Rollbar] Scheduling payload

Here's a couple issues you may run into:

You may need to rebuild your apps for them to start piping to logspout. dokku ps:rebuildall When you re-deploy your apps, the heroku-like program names will be lost. To fix this run dokku logspout:stop && dokku logspout:start. Hopefully this issue will be fixed soon. docker ps returns a daemon error. Run sudo usermod -aG docker ubuntu && sudo reboot as ubuntu.

Continue Reading

3 Email Efficiency Tips

I wanted to share a couple quick email efficiency tips that have been helpful over the last couple weeks.

When scheduling a meeting, instead of asking "What time would be good to meet?" suggest a exact time and day with two alternatives that work for you. Also provide a link to your full public calendar with event details hidden (I use ScheduleShare for this). For example:

Would 2-2:30pm on Thursday the 12th work for you? If not, would Wednesday at 10:00am or Thursday at 11:00am work? Feel free to suggest another time that works best for you; here's my full schedule: LINK

Also, when you've decided on a meeting time, create a meeting invite and include a conference line as the location of the calendar event:

I just sent over an invite with a conference line. Feel free to suggest another conference tool.

This avoids another round of back and forth deciding which conferencing tool to use, and then asking for Skype/G+/etc usernames. I've found that for most one-off meetings, a simple analog phone line works best: it doesn't require any software, and eliminates the always-present audio issues that exist with software-based conferencing solutions.

If you are responding to a unsolicited request, don't feel the need to write a full in-depth response. Provide some value, and then ask the correspondent for some additional information that will require a thoughtful response on their end. I'm a people pleaser and a perfectionist, it's been a discipline for me not to help anyone who comes asking for help. Often times, I've found that when pressed for more information about the problem someone is facing, I don't receive a reply and end up saving myself a lot of time writing a thoughtful response.

Continue Reading

Steps to Refocus, Regroup, and Get Productive

After a series of unrelated meetings or going through a full inbox, it's easy to feel scattered, unfocused, and distracted.

This has happened to enough times, that I've written out an instruction manual for myself to enable me to quickly get refocused and back on the right track:

Record everything in your head. Dump all tasks into your favorite task manager (ToDoist is mine!). Don't attempt to label, prioritize, or organize tasks. Collect scattered tasks. Get all tasks, ideas, or unorganized bits of information in a centralized location. Run through any written information and transcribe any actionable items. I'd recommend setting up a "paper inbox" to throw all paper, notes, mail, etc so it's easy to process. Look through missed calls and listen to all voicemails and record any tasks. Look through all text messages and record any tasks. Run through your email and move any tasks to your todo list. Todoist has a feature that enables you to email tasks into your todoist inbox; this is super helpful for quickly extracting action items from your inbox. Prioritize. Run through the list and prioritize all tasks. Eliminate. Is there anything you can: Delay. Does it really need to be done now? Automate. Is there a recurring task that you could use zapier or another service to completely automate? Outsource/Delegate. Use a virtual assistant, fiverr, etc. Think you can't afford a virtual assistant? Calculate how much your time is worth, realize that your time is worth more than $20/hour, and then start getting comfortable paying others to do things you don't enjoy. Delete. If you've committed to too much, try to get out of the commitment. This is super hard for me (I'm a people pleaser), but it's so essential to making sure you make good on the commitments you've already made. Rescan your list. Is there anything you can hand off to someone who will do a 80% ok job? "Good enough" can be "great" if you didn't spend any time on it! Make sure your list is manageable. You know how much you can get done in a day, make sure your todo list for the day can be accomplished without sacrificing your margin. Apologize & communicate delays. If you're slammed, most likely you are going to be delayed on getting some tasks done. Make sure that anyone who is depending on you knows about the delay and understands what the revised timeline looks like.

Although it takes time to run through this process, it's been an enormous help. What works for you? How do you get refocused when you feel unfocused?

Continue Reading

Action Steps for Doubling Morning Productivity

Here's my system for engineering an incredibly productive distraction-free work block from when you wake up to 12:30pm. Although I don't follow this routine all of the time, when I do I've hugely increased my ability to cut through the urgent and ensure that most important tasks for the day get completed.

Download Focus App. Setup a weekly schedule to block all distractions from 4:30am to 12:30pm on all weekdays. If you want to impress your friends, block all distractions everyday. In addition to the stock list of websites, here are some applications I block: Mail. If you use the gmail web interface, download MailPlane so you can easily block it. If you need to send an email during your distraction-free time block, just open up the browser. Messages. I hate text messages even more than email. There are no open standards, which means there is no tooling (or innovation!) around separating context and managing distraction. I've resorted to blocking the entire app and scheduling a recurring task (via ToDoist) to batch respond to messages. This doesn't work completely: I still need to respond to some people in real-time. Still not happy with this piece of the puzzle: I would love an app that whitelisted specific contacts as important and provided a distraction free interface to respond. Twitter. The exception: Slack. I keep Slack open although it could be a huge distraction. Too much business-critical communication happens here. Most routers allow you to setup internet access schedules for specific devices on your network. I've blocked wifi on my phone from 8pm-12:30pm. This adds a little bit of friction to using my phone. Put your phone on the other side of the room, or in a place where you can't touch it. This adds additional friction to checking your messages/notifications, even when your phone is on "Do Not Disturb" mode. Setup parental controls on your iPhone: block all social networks or distracting news sites. Setup a "Do Not Disturb" schedule on your phone from 8pm-12:30pm. Here's a quick solution to the "I need to be available in case someone really needs to get ahold of me!" problem. Create a group in the Apple address book with a list of people who you need to be responsive to (for family, personal, or business reasons). Then, whitelist this group in your "Do Not Disturb" settings. However, the above trick only whitelists calls, not texts. I've slowly been communicating that phone and slack are the best channels to grab me. If it's not important enough to call, or if you are not in a Slack group that I monitor, then it can't be urgent enough to break my morning concentration time. Setup RescueTime to track your time usage. This gives me weekly metrics to guide me to areas of improvement. Move the mail app on your phone to the last screen. This makes checking email on your phone a deliberate choice instead of a impulsive action. Disable all notifications on your phone. Anything that you don't need to know about right then should be sent to email where it can be handled in batches. Here are some notifications which I've disabled: Facebook Messenger, Twitter, email, LinkedIn, DropBox. iMessage and Calendar are the only notifications which I allow on my phone.

Let me know what you think: I'd love to hear any critique or tips you've found to be effective.

Continue Reading

Depositing Transaction Records in NetSuite

Moving transactions in NetSuite from "Not Deposited" to "Deposited" is not straightforward when using NetSuite SuiteTalk.

You need to ensure that undocumented requirements for each record type are met. After you've validated that your records are properly configured, you can include them in a new deposit using the following structure:

ns_deposit = ns_deposit.payment_list << { deposit: true, # internalId of a CashSale, CustomerPayment, CustomerRefund, CustomerDeposit, etc id: 123, # the amount is not required if the currency of the payment is the same as the currency # of the bank account that the deposit is posting to payment_amount: 100.0 } ns_deposit.add

Note that you don't need doc, type, or any of the other fields available on the a DepositPayment item. It's also important to note that the other fields on the DepositPayment don't actually effect how NetSuite handles the referenced transaction record. deposit, id, and payment_amount are the only fields that matter when referencing a NetSuite transaction on a deposit.

Continue Reading

Using Browser Sync with Rails and Local SSL

I've always been a fan of LiveReload since I first saw it years ago. However, the rack-livereload gem hasn't been updated in a long while, and the livereload parent project seems to have died out too. rack-liveload doesn't support reloading on local HTTPS connections, which was a requirement on a recent project—I needed a new tool.

BrowserSync came to the rescue.

In most cases, using BrowserSync's proxy mode to reverse proxy a rails app running via a *.dev pow domain works well. For this particular project, there were a couple of specific issues:

Many pages on the project force HTTPS. A common pattern is to disable HTTPS in development: I prefer to keep the development and production delta as small as possible, and keep any HTTPS redirects enabled in production active in development as well. The site used oauth for local user authentication. Each oauth redirect URL needs to be specified in each oauth provider's admin. This is a pain to manage, and becomes more a pain when a reverse proxy with a unique domain is used sometimes. The SSL certs served by tunnelss were not valid for the BrowserSync server causing browsers to throw invalid certificate errors.

After a bit of digging, I found a way to use my existing *.dev setup with BrowserSync. Here's how:

Setup Apache + Pow + Tunnelss

Read through this blog post to setup your development environment to serve SSL.

Point Browser Sync to Tunnelss-generated SSL Certificates

BrowserSync has a bunch of configuration options. Some are inaccessible via the command line interface and need to be specified in a javascript file passed to the command line app.

Here's how to configure BrowserSync to use the tunnelss-generated SSL certs to serve the secure web sockets it uses for syncing:

// module.exports = { // "logLevel": "debug", "files": [ '**/*.css*', '**/*.html*', '**/*_controller.rb', '**/*.md' ], "https": { "key": process.env['HOME'] + "/.tunnelss/key.pem", "cert": process.env['HOME'] + "/.tunnelss/server.crt" } };

Throw this configuration file in bs-config.js in the root level of your project and start the BrowserSync server with:

browser-sync start --config bs-config.js Connect Your Rails (or Sintra) App to BrowserSync

To connect your application to the BrowserSync server, throw this simple javascript tag in your layout:

<% if Rails.env.development? %> <script type='text/javascript' id="__bs_script__">//<![CDATA[ document.write("<script async src='https://HOST:3000/browser-sync/browser-sync-client.js'></script>".replace("HOST", location.hostname)); //]]></script> <% end %>

Note that the above snippet does not define a browser sync version in the javascript. This makes this code a bit more resilient to browser sync updates.

I prefer this simple JS snippet to a middleware (like rack-livereload) that inserts the JS snippet into your site automatically. Keeping this code in your layout makes it easy to upgrade the code in the future, and is more declarative to new developers working on the project.

Continue Reading

Using HTTPS Locally: Pow, SSL, Rails, and Apache

Using HTTPS local development or testing environments is a hassle. A common practice is to disable SSL in development, but I've had too many cases of minor bugs creeping in when routing works differently between environments. I also don't like manually having to let my browsers know that a self-signed certificate is valid.

Here's the configuration that I use to easily add https support on my development machine to any application served—or proxied—through port 80.


I use Pow as my development rails server. However, because I often work with other languages as well, I run Apache (you could just as easily use nginx) and reverse proxy Rails application requests to Pow.

To do this, you'll need Pow to bind to a port that is not port 80. Here's how to configure Pow to use port 88.


I use nginx in production environments, but I've been using Apache on my local development machines for years and just haven't gotten around to changing my dotfiles to use nginx. You could just as easily use nginx here.

In any case, here's what I was looking to achieve in my apache config:

Throw error logs in ~/Sites/logs Support vhosts + domains Access WordPress sites using Access Rails applications using These requests need to reverse proxy to pow on port 88. Support proxying *.test domains to a customized port. Having a *.test domain allows you to run SSL-enabled integration tests for a multi-domain Rails application against the rspec-run rails server. Checkout my series on rails testing for more details on why this is important.

I was able to get all of this working with a single file thrown in /etc/apache2/other.

Rails: Tunnelss to the Rescue

Tunnelss is a little gem that brings it all together. It looks at your pow config, generates self-signed certificates for those domains, and adds that certificate to your keychain so your browsers accept the self-signed certificate. Brilliant.

The only downside here is you must keep tunnelss running in the background. Right now, the project doesn't have a launchd plist available. So, for now, you have to start the application manually on each system restart.

Continue Reading

How to Refund a Customer Deposit using NetSuite SuiteTalk

When refunding a NetSuite CustomerRefund or CashSale you'll add it to apply_list:

refund.apply_list = { apply: [ { doc: 123, # internalId of the CustomerRefund apply: true } ] }

Refunding a CustomerDeposit works a bit differently. If you inspect the SuiteTalk XML response for a CustomerRefund created for a CustomerDeposit it will appear under the apply_list XML tag. If you use that XML tag when creating the CustomerRefund you'll get the following error:

Unable to find a matching line for sublist apply with key

Instead, you need to use the deposit_list tag:

refund.deposit_list = { customer_refund_deposit: [ { doc: 123, # internalId of CustomerDeposit apply: true } ] }

Continue Reading

7 Categories to Help Guide Your 2016 Goals

Results follow clarity, and clarity comes from clear definition. Writing is the best way to clearly define your thoughts (one of the reasons you should start a blog!).

Here are my goal categories, in order of long-term priority. Explicitly defining these categories has helped me during my goal planning process to identify which categories needs the most attention this upcoming year, and ensure that my goals align against these categories:

Spiritual Marriage Health Intellectual Financial Work Relationships

This ordering doesn't mean that work won't come before my intellectual life (i.e. reading, conferences, learning, etc). It does mean, for me, over the long-term I need to ensure that my intellectual life is made a priority over work success.

Although simple, the exercise of defining and prioritizing the different categories of your life, is incredible important.

Continue Reading