Fixing Bundler Issues After a macOS Sierra Upgrade

I recently upgraded my laptop, and with that came setting up a fresh dev machine. With a fresh install always comes with a set of development configuration problems. Here’s some tips of resolving ruby bundler issues in MacOS Sierra.

These tips assume you are using dotfiles that look similar to these. Also, I prefer fresh installs, so I never pull config from my old machine/OS. These fixes may not work if you are migrating configuration from a previous OS.

For any of the fixes outlined below:

Adjust the version strings based on your project If you do not have bundler’s shared gems disabled, then you can omit the --install-dir directives.

An error occurred while installing pg (0.18.4), and Bundler cannot continue.

gem install --install-dir vendor/bundle pg -v 0.18.4 -- --with-pg-config=/Applications/Postgres.app/Contents/Versions/9.6/bin/pg_config

Alternatively, you can configure this setting with bundle config build.pg.

An error occurred while installing nokogiri (1.6.8), and Bundler cannot continue. An error occurred while installing mysql2 (0.3.20), and Bundler cannot continue. xcode-select --install

An error occurred while installing capybara-webkit (1.11.1), and Bundler cannot continue.

brew install qt55 brew link --force qt55

Then, I needed to manually edit mkspecs/features/Mac/default_pre.prf. More details here.

error occurred while installing therubyracer (0.12.1), and Bundler cannot continue

gem install --install-dir vendor/bundle libv8 -- --with-system-v8 gem install --install-dir vendor/bundle therubyracer -v 0.12.1

rubygems/core_ext/kernel_require.rb:54:in `require’: cannot load such file — bundler/setup (LoadError)

This is caused by spring coupled with a new version of bundler. Regenerate your binstubs:

bundle exec spring binstub --all

Enter passphrase for /Users/USER/.ssh/id_rsa

If you copy your public + private key from your old machine/OS, and used a passphrase to secure it (which you should be doing!), you’ll need to run this command to avoid reentering your passphrase every time you use the key:

ssh-agent bash ssh-add # in some cases (capistrano deployment, for example) you'll need to do this: ssh-add -K ~/.ssh/id_rsa # Also, in some cases, this helped: edit ~/.ssh/config Host * UseKeychain yes

An error occurred while installing eventmachine (1.0.4), and Bundler cannot continue.

Specifically, I got this build error:

Gem::Ext::BuildError: ERROR: Failed to build gem native extension. ~/.rbenv/versions/2.1.6/bin/ruby extconf.rb checking for rb_trap_immediate in ruby.h,rubysig.h... no creating Makefile make "DESTDIR=" clean make "DESTDIR=" compiling binder.cpp In file included from binder.cpp:20: ./project.h:112:10: fatal error: 'openssl/ssl.h' file not found #include <openssl/ssl.h> ^ 1 error generated. make: *** [binder.o] Error 1 make failed, exit code 2

Here’s the fix (inspired by this post) that fixed the issue:

gem install --install-dir vendor/bundle eventmachine -v '1.0.4' -- --with-cppflags=-I/usr/local/opt/openssl/include

/.rbenv/versions/2.2.3/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require’: cannot load such file — bundler/setup (LoadError)

This error was being caused by spring when attempting to run bundle exec rails console. You can verify that this is the case by omitting spring with export DISABLE_SPRING=true and seeing if rails console runs just fine.

To fix this issue, you need to update your binstubs:

bundle exec rake rails:update:bin

Continue Reading

Notes on NetSuite’s Gift Certificate Record

The gift certificate functionality in NetSuite is much more limited than other functionality in NetSuite. It’s also the only option for implementing store credit, electronic, and physical gift cards in NetSuite. If you need to integrate with the gift certificate system in NetSuite, here are some notes to help you along the way:

Gift certificates are not created directly. They are are created by creating a “Gift Certificate Item” and using that item in a SalesOrder. Gift certificates can be created without an associated posting transaction (Sales Order) by importing a CSV. However, you cannot create a standalone gift certificate via SuiteTalk. You must create a posting transaction (Invoice/CashSale) with a gift certificate item in order to create a gift certificate via SuiteTalk. Technically, a non-posting transaction (Sales Order) will create a gift certificate record. However, the gift certificate cannot be used until it is associated with a posting transaction. The “Amount Remaining” field is nil if the gift card has been created by a non-posting transaction by has not been filled/purchased. In other words, if amount remaining is nil the user should not be able to use this gift card on an ecommerce site. Gift certificate codes can be a maximum of 9 characters. It’s important that external systems respect this limit if NetSuite is not the system-of-origin for gift card codes. The gift certificate code can be blank. It’s hard to tell exactly how this can occur because there is no system log, but I believe it occurs when gift certificates are imported. There is a custom field list advances search option for gift certificates, but strangely enough custom fields cannot be added to the gift certificate item. The system log is not available for a gift certificate item The last modified date is not shown in the UI You cannot search gift certificates by the last modified date, although it is available in the SuiteTalk response. The last modified date is not updated when a gift card is used. The best way to search for updated gift cards is using a transaction search. A gift certificate is not tied to a specific customer, although in some areas of the UI it does seem as though The gift certificate record does not contain a reference to the customer that purchased or used the gift card. You must search for transactions associated with the gift certificate, then pull a customer reference from the associated transaction. Use the transaction search’s giftCertificate field to retrieve all transactions that are connected with the gift certificate. Note that this includes the non-posting transaction that may have created the gift certificate record, along with any other transactions that have effected the gift card balance over time. You’ll need to filter out the exact transactions you are looking for.` If you need to automatically generate codes that play nicely with other systems (3PL, ecommerce, etc) the best approach is create a before save via SuiteScript to call out to an external service that coordinates gift card code generation. NetSuite does provide the option to generate gift certificate codes for you. However, if you have other systems (such as an eCommerce platform), you won’t be able to push gift certificates into NetSuite using the externally generated code if this option is enabled. If a transaction’s line item contains the giftCertCode field, then the line item is a GiftCertificateItem. If a transaction’s header-level giftCertApplied field is less than zero, then a gift certificate was used as part of the payment. If you search for transactions with giftCertificate not empty ItemFulfillments will be returned in the results, although there is no reference to a gift certificate record on that transaction. A gift card notification email will be sent to the email in the “To” field. There is no way to disable this feature. Example Gift Card Transaction Search

Here’s an example transaction search using the NetSuite ruby bindings for any transactions that use a gift card:

NetSuite::Records::SalesOrder.search( basic: [ { field: 'giftCertificate', operator: 'notEmpty' } ] )

Note that not just SalesOrders are returned. Any transaction associated with a gift certificate is returned in this search. To restrict to a specific transaction type, using a type = _salesOrder search criteria.

Enhancements Requested

I requested enhancements for the more critical gift card limitations I found through SuiteIdeas (vote on them if you can!):

116981 Ability to disable gift certificate emails 411408 Ability to search by gift certificate’s last modified date 176418 Gift certificate custom fields can’t be viewed in SuiteTalk response 159896 Cannot create custom fields on gift certificate

Continue Reading

Finding the NetSuite Deposit Associated with a Payment or Refund

Determining which NetSuite deposit record is associated with a customer payment, customer refund, cash payment, cash refund, etc using SuiteTalk is not straightforward. It’s also challenging to determine if a refund record is still available to be deposited. Payment transactions have a status field which indicates if a record has been deposited or not, but refund records do not have this field.

In order to determine if a refund is deposited, you need to search for an associated deposit. Here’s a sample implementation using the NetSuite ruby bindings:

ns_customer_refund_id = 2158604 search = NetSuite::Records::Deposit.search( criteria: { basic: [ { field: 'type', operator: 'anyOf', value: [ '_deposit' ] }, { field: 'appliedToTransaction', operator: 'anyOf', value: [ NetSuite::Records::RecordRef.new(internal_id: ns_customer_refund_id) ] }, ] }, preferences: { page_size: 10 } ) ns_deposit = search.results.first # deposit needs to be refreshed to include body fields ns_deposit.refresh Linking to the Associated Deposit in the NetSuite GUI

Another way to solve this problem is creating a custom field on the refund record and use a formula to pull the associated deposit from the refund. NetSuite stores a reference to the deposit in their refund DB table that is not exposed to SuiteTalk or SuiteScript. Here’s more details on this pulled from a conversation with NetSuite support:

I created a custom body field on Customer Refund page to display the associated Bank Deposit Number as steps listed below.

Navigate to Customization > Lists, Records, & Fields > Click on Transaction Body Fields > New.

Enter a Label for the custom field. For example, Bank Deposit Number (Test).

Type field: select ‘Free-Form Text’

Store Value checkbox > Unchecked.

Under Applies To tab > Check ‘Sales’

Under Display tab > Subtab > I selected ‘Main’

Under Validation&Defaulting tab > Default Value field > Enter ‘{deposittransaction.number} ‘

Save.

Navigate to Customization > Forms > Transaction Forms > Edit the Customer Refund transaction form in concern/in use > Under Screen Fields tab > Main Subtab > Make sure that the above custom field created is checked with ‘Show’ checkbox > Save.

Now, if you open/view an existing Customer Refund transaction that is associated with a Bank Deposit, the Bank Deposit number will show automatically.

Searching for the Associated Deposit Using a Transaction Search

Here’s an example of how to search for the associated deposit using the NetSuite GUI. Note that the “Applied To Transaction” field requires that you prefix the payment transaction ID with “Payment #” or you won’t be able to find a match in the search creation GUI.

Here’s a direct link to open up a deposit transaction search.

Continue Reading

Using NetSuite’s Token Based Authentication with SuiteTalk

NetSuite’s OAuth is very different from the standard oauth flow: setting up a user for token based auth is very cumbersome. It requires digging around in the NetSuite GUI, creating roles, and copy/pasting various keys.

Why use token based authentication? The alternative is email + password based authentication. This method works fine, but passwords expire every six months; resetting passwords every six months is a huge pain for a SAAS product that integrates with NetSuite. Plus, email + password auth is much less secure (an attacker can login to the GUI with a email and password).

Here’s a guide to getting setup with token based authentication. Note that you must be using a SuiteTalk API versions greater than 2015_2.

1. Create a Integration Record

The integration record identifies the application in NetSuite’s system.

Visit the integrations page or global search for page:integrations Create a integration record if none exists. After you create the record you will need to copy/paste the consumer key and consumer secret to your secrets file. Name: Your-Application-Name Authentication: Token-Based Authentication State: Enabled If the integration record already exists, but you don’t have the consumer key and consumer secret, edit the record, then press “Reset Credentials”. 2. Enable Token Based Authentication Setup > Company > Setup Tasks > Enable Features > SuiteCloud > Manage Authentication Make sure “Token Based Authentication” is enabled Save

If this feature is not enabled, you will not see the permissions required in the next step.

3. Create a Token Role

Strangely enough, the administrator does not have token permissions by default. If you do not create a token role and assign it to your administrator, you will get a “Login access has been disabled for this role.” error when creating a token.

Global search for page:role, then choose “New Role” Navigate to “Permissions > Setup” and add the following permissions: User Access Token: Full Access Token Management: Full Web Services: Full 4. Add Token Management Permissions Global search for page:employees Edit your employee record Navigate to “Access > Roles” and add the token auth role you just created 5. Create Access Tokens Global search for page: tokens New Access Token Select the application and role we created earlier, then press save. Copy/past the token ID and token secret to your secrets file.s 6. Configure Your Client

Here’s how to setup the netsuite ruby client with token based authentication:

NetSuite.configure do reset! account ENV['NETSUITE_ACCOUNT'] consumer_key ENV['NETSUITE_CONSUMER_KEY'] consumer_secret ENV['NETSUITE_CONSUMER_SECRET'] token_id ENV['NETSUITE_TOKEN_ID'] token_secret ENV['NETSUITE_TOKEN_SECRET'] end

Continue Reading

Fixing Invalid DATABASE_URL Reference on CircleCI

I’ve become a huge fan of CircleCI—it’s the best CI tool for small-medium projects that I’ve found. Recently I ran into an issue setting up a new rails project with Postgres as the database backend with circle:

export RAILS_ENV="test" export RACK_ENV="test" bundle exec rake db:create db:schema:load --trace ** Invoke db:create (first_time) ** Invoke db:load_config (first_time) ** Execute db:load_config rake aborted! NameError: Cannot load `Rails.application.database_configuration`: uninitialized constant DATABASE_URL

However, when I looked at my database.yml the constant DATABASE_URL was clearly not being used. DATABASE_URL was properly being used as the lookup string for an ENV reference:

# config/database.yml test: url: <%= ENV['TEST_DATABASE_URL'] %>

I was stumped for a while until I ssh’d into circle and noticed that somehow the single quotes were stripped from DATABASE_URL:

# cat config/database.yml test: url: <%= ENV[DATABASE_URL] %> adapter: postgresql database: circle_ruby_test username: ubuntu host: localhost

Circle’s build inference process involves generating a database.yml file which uses the original database.yml file to some extent. Some part of the process involves stripping single quotes, which is what was causing the build breakage. Make sure that you use double quotes instead of single quotes in database.yml when using CircleCI.

Automating Staging Deploys to Staging on CI Build Success

Here’s how to automatically push new builds (and run database migrations) to a heroku-hosted app using circle CI:

deployment: staging: branch: master commands: - heroku maintenance:on --app the-app-name - git push git@heroku.com:the-app-name.git $CIRCLE_SHA1:master - heroku run rake db:migrate --app the-app-name - heroku maintenance:off --app the-app-name

Continue Reading

Sending Dokku Container Logs to Papertrail

I’m a huge fan of Heroku. Although it’s expensive compared to a raw DigitalOcean droplet or EC2 instance, it’s so easy and it just works.

However, there are some use-cases that Heroku doesn’t handle well. Business integration systems (eCommerce NetSuite integration, for example) often require handling requests that may take longer than the 30 second request timeout.

Dokku is the next-best alternative to using heroku (if you don’t have enormous scaling needs: dokku apps are restricted to a single host). Here’s how to push your dokku app logs running to PaperTrail, or another syslog-compatible cloud hosting provider (I’m sure this approach would work for Loggly or CloudWatch).

Install dokku-logspout: dokku plugin:install https://github.com/michaelshobbs/dokku-logspout.git Point logspout to your PaperTrail syslog endpoint: dokku logspout:server syslog+tls://logs.papertrailapp.com:12345. Make sure your PaperTrail endpoint is configured to accept TLS connections. Install dokku-hostname: dokku plugin:install https://github.com/michaelshobbs/dokku-hostname.git dokku-hostname. This will ensure that the host in your PaperTrail logs use the heroku-like name of your worker instead of the docker container ID. Run dokku logspout:start and use logspout:stream to ensure that your logs are being aggregated properly.

The resulting log output should look something like:

Apr 15 14:49:28 host.com pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:167:in `stats' Apr 15 14:49:28 host.com pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:127:in `process' Apr 15 14:49:28 host.com pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:79:in `process_one' Apr 15 14:49:28 host.com pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:67:in `run' Apr 15 14:49:28 host.com pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:16:in `watchdog' Apr 15 14:49:28 host.com pipeline.worker.1: /app/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:24:in `block in safe_thread' Apr 15 14:49:28 host.com pipeline.worker.1: 11 TID-os97lykxo INFO: [Rollbar] Scheduling payload

Here’s a couple issues you may run into:

You may need to rebuild your apps for them to start piping to logspout. dokku ps:rebuildall When you re-deploy your apps, the heroku-like program names will be lost. To fix this run dokku logspout:stop && dokku logspout:start. Hopefully this issue will be fixed soon. docker ps returns a daemon error. Run sudo usermod -aG docker ubuntu && sudo reboot as ubuntu.

Continue Reading

3 Email Efficiency Tips

I wanted to share a couple quick email efficiency tips that have been helpful over the last couple weeks.

When scheduling a meeting, instead of asking “What time would be good to meet?” suggest a exact time and day with two alternatives that work for you. Also provide a link to your full public calendar with event details hidden (I use ScheduleShare for this). For example:

Would 2-2:30pm on Thursday the 12th work for you? If not, would Wednesday at 10:00am or Thursday at 11:00am work? Feel free to suggest another time that works best for you; here’s my full schedule: LINK

Also, when you’ve decided on a meeting time, create a meeting invite and include a conference line as the location of the calendar event:

I just sent over an invite with a conference line. Feel free to suggest another conference tool.

This avoids another round of back and forth deciding which conferencing tool to use, and then asking for Skype/G+/etc usernames. I’ve found that for most one-off meetings, a simple analog phone line works best: it doesn’t require any software, and eliminates the always-present audio issues that exist with software-based conferencing solutions.

If you are responding to a unsolicited request, don’t feel the need to write a full in-depth response. Provide some value, and then ask the correspondent for some additional information that will require a thoughtful response on their end. I’m a people pleaser and a perfectionist, it’s been a discipline for me not to help anyone who comes asking for help. Often times, I’ve found that when pressed for more information about the problem someone is facing, I don’t receive a reply and end up saving myself a lot of time writing a thoughtful response.

Continue Reading

Steps to Refocus, Regroup, and Get Productive

After a series of unrelated meetings or going through a full inbox, it’s easy to feel scattered, unfocused, and distracted.

This has happened to enough times, that I’ve written out an instruction manual for myself to enable me to quickly get refocused and back on the right track:

Record everything in your head. Dump all tasks into your favorite task manager (ToDoist is mine!). Don’t attempt to label, prioritize, or organize tasks. Collect scattered tasks. Get all tasks, ideas, or unorganized bits of information in a centralized location. Run through any written information and transcribe any actionable items. I’d recommend setting up a “paper inbox” to throw all paper, notes, mail, etc so it’s easy to process. Look through missed calls and listen to all voicemails and record any tasks. Look through all text messages and record any tasks. Run through your email and move any tasks to your todo list. Todoist has a feature that enables you to email tasks into your todoist inbox; this is super helpful for quickly extracting action items from your inbox. Prioritize. Run through the list and prioritize all tasks. Eliminate. Is there anything you can: Delay. Does it really need to be done now? Automate. Is there a recurring task that you could use zapier or another service to completely automate? Outsource/Delegate. Use a virtual assistant, fiverr, etc. Think you can’t afford a virtual assistant? Calculate how much your time is worth, realize that your time is worth more than $20/hour, and then start getting comfortable paying others to do things you don’t enjoy. Delete. If you’ve committed to too much, try to get out of the commitment. This is super hard for me (I’m a people pleaser), but it’s so essential to making sure you make good on the commitments you’ve already made. Rescan your list. Is there anything you can hand off to someone who will do a 80% ok job? “Good enough” can be “great” if you didn’t spend any time on it! Make sure your list is manageable. You know how much you can get done in a day, make sure your todo list for the day can be accomplished without sacrificing your margin. Apologize & communicate delays. If you’re slammed, most likely you are going to be delayed on getting some tasks done. Make sure that anyone who is depending on you knows about the delay and understands what the revised timeline looks like.

Although it takes time to run through this process, it’s been an enormous help. What works for you? How do you get refocused when you feel unfocused?

Continue Reading

Action Steps for Doubling Morning Productivity

Here’s my system for engineering an incredibly productive distraction-free work block from when you wake up to 12:30pm. Although I don’t follow this routine all of the time, when I do I’ve hugely increased my ability to cut through the urgent and ensure that most important tasks for the day get completed.

Download Focus App. Setup a weekly schedule to block all distractions from 4:30am to 12:30pm on all weekdays. If you want to impress your friends, block all distractions everyday. In addition to the stock list of websites, here are some applications I block: Mail. If you use the gmail web interface, download MailPlane so you can easily block it. If you need to send an email during your distraction-free time block, just open up the browser. Messages. I hate text messages even more than email. There are no open standards, which means there is no tooling (or innovation!) around separating context and managing distraction. I’ve resorted to blocking the entire app and scheduling a recurring task (via ToDoist) to batch respond to messages. This doesn’t work completely: I still need to respond to some people in real-time. Still not happy with this piece of the puzzle: I would love an app that whitelisted specific contacts as important and provided a distraction free interface to respond. Twitter. The exception: Slack. I keep Slack open although it could be a huge distraction. Too much business-critical communication happens here. Most routers allow you to setup internet access schedules for specific devices on your network. I’ve blocked wifi on my phone from 8pm-12:30pm. This adds a little bit of friction to using my phone. Put your phone on the other side of the room, or in a place where you can’t touch it. This adds additional friction to checking your messages/notifications, even when your phone is on “Do Not Disturb” mode. Setup parental controls on your iPhone: block all social networks or distracting news sites. Setup a “Do Not Disturb” schedule on your phone from 8pm-12:30pm. Here’s a quick solution to the “I need to be available in case someone really needs to get ahold of me!” problem. Create a group in the Apple address book with a list of people who you need to be responsive to (for family, personal, or business reasons). Then, whitelist this group in your “Do Not Disturb” settings. However, the above trick only whitelists calls, not texts. I’ve slowly been communicating that phone and slack are the best channels to grab me. If it’s not important enough to call, or if you are not in a Slack group that I monitor, then it can’t be urgent enough to break my morning concentration time. Setup RescueTime to track your time usage. This gives me weekly metrics to guide me to areas of improvement. Move the mail app on your phone to the last screen. This makes checking email on your phone a deliberate choice instead of a impulsive action. Disable all notifications on your phone. Anything that you don’t need to know about right then should be sent to email where it can be handled in batches. Here are some notifications which I’ve disabled: Facebook Messenger, Twitter, email, LinkedIn, DropBox. iMessage and Calendar are the only notifications which I allow on my phone.

Let me know what you think: I’d love to hear any critique or tips you’ve found to be effective.

Continue Reading

Depositing Transaction Records in NetSuite

Moving transactions in NetSuite from “Not Deposited” to “Deposited” is not straightforward when using NetSuite SuiteTalk.

You need to ensure that undocumented requirements for each record type are met. After you’ve validated that your records are properly configured, you can include them in a new deposit using the following structure:

ns_deposit = NetSuite::Records::Deposit.new ns_deposit.payment_list << { deposit: true, # internalId of a CashSale, CustomerPayment, CustomerRefund, CustomerDeposit, etc id: 123, # the amount is not required if the currency of the payment is the same as the currency # of the bank account that the deposit is posting to payment_amount: 100.0 } ns_deposit.add

Note that you don’t need doc, type, or any of the other fields available on the a DepositPayment item. It’s also important to note that the other fields on the DepositPayment don’t actually effect how NetSuite handles the referenced transaction record. deposit, id, and payment_amount are the only fields that matter when referencing a NetSuite transaction on a deposit.

Continue Reading