• Better Golang Errors With errors.Wrap

    Libraries shouldn’t log. Logging takes resources and should occur as rarely as possible. If an error occurs deep in your program, you don’t want to immediately log it, bubble the error up the stack, and then log the same error again but with additional context. Thus, you should try to log as close to the UI as possible. If you’re writing a web service, only log in the handler before you send a response. The code for accessing the database is a dependency and should be in its own package. We treat it like a library and only concern ourselves with its API.

    But… having more context would be helpful. Even though it’s wasteful, logging an error twice reveals more about the current code path when the bug occurred.

    One way to provide more context is to have very specific messages for each log line that are never repeated. For example…

    const (
      UnauthorizedUserLogMsg                string = "Unauthorized request"
      UnmarshallingRequestErrorLogMsg       string = "Error unmarshalling request body"
      JSONMarshallingResponseErrorLogMsg    string = "Error marshalling json response body"
      JSONAPIMarshallingResponseErrorLogMsg string = "Error marshalling jsonapi response body"
      DatastoreErrorLogMsg                  string = "Error returned by the datastore"
    )
    
    func (handler *Handler) CompaniesGetAll(resp http.ResponseWriter, req *http.Request) {
      companies, err := handler.Datastore.CompaniesGetAll(limit, offset)
      if err != nil {
        handler.Logger.Error().Err(err).Msg(DatastoreErrorLogMsg)
        http.Error(rw, "Internal Server Error", http.StatusInternalServerError)
        return
      }
    
      // Handle a normal response
      // ...
    }

    If the Datastore package is returning a Postgresql error, that error could have come from any request to the database in CompaniesGetAll, and the error messages from the Go Postgresql adapter are especially obscure.

    Enter github.com/pkg/errors, a package that lets you wrap your errors with additional context. Calling errors.Wrap(err, message) returns a struct that conforms to the errors interface where message is prepended to the error. Everytime an error is returned, you can add context to that error message.

    My preferred way to use errors.Wrap is to keep it concise. The message added should have the format “verbing noun”.

    var count int
    err := db.DB.QueryRow("SELECT COUNT(*) FROM companies;").Scan(&count)
    if err != nil {
      return 0, errors.Wrap(err, "querying company count")
    }

    Now your errors have more context AND you are not logging in your packages.

    1. I will allow an exception for packages that start their go routines that run alongside your app. If your package needs to log information, please please please let the user pass in an io.Writer. Most loggers support the io.Writer interface, so your user's can use their logging library and format. If your logger writes JSON and you use Datadog's tracer, error messages from Datadog aren't going to match your pretty formatted logs
    2. I wish there was a standard interface for logging that gave you log levels.
    3. Secret trick: if your apps performance is degrading, disable logging. Verizon disables logging when new iPhones launch to prevent themselves from being DDoS'd by thirsty Apple users.
    4. Zerolog, my logger of choice, lets you use Diodes, a ring buffer, for writing logs. If logs are written to Diodes faster than stdout can read them, logs get dropped until stdout catches up.
  • Interviewing Like You're Dating

    Interviewing and dating are strangely similar. I assumed they’d magically work themselves out without additional effort on my part! I kid… I kid…

    Expect to Be Rejected a Lot

    It’s inevitable, but being rejected is the only way to build up a tolerance to rejection.

    The Best Leads Are Through Your Social Circle

    I’ve never lived Patrick McKenzie’s scenario where the company has decided they’re going to hire you and the interview is just a formality. Having someone who’s worked with you refer you to a company lets you skip some steps. Sometimes it’s only the HR screening. Sometimes you jump straight to an onsite interview.

    Be Legitimately Interested

    This should be obvious, but 2008 Jim needed to learn this. Don’t apply to a company unless you’re interested. Every interview process is going to ask you why you’re interested in [company]. You should know before you apply. If you have to practice it in the mirror, maybe you shouldn’t be applying.

    Sometimes this will come up when they ask you how you heard about [company]. Once I was able to honestly reply that I read their engineering blog and fell in love. They rejected me.

    Alternatively, it is very very hard to answer this question if the real answer is that [company] uses my favorite tech stach, pays above market, and the people are decent human beings.

    Ask Questions that Start Conversations

    One sided discussions are boring. Both you and the interviewer want to ask questions that start conversations.

    2008 Jim didn’t have questions to ask companies. At my early jobs developers had no input on the tech stack or build and deployment pipeline. I was less engaged in interviews, and interviewers probably mistook it for disinterest.

    My current favorite questions are about testing and build pipelines. Having automated builds and deployment is more and more common. It might be a very boring question in five years.

    My ideal interview would be talking shop about interesting bugs and what crazy ways [company] is dysfunctional (Every company is in some way). Alas, here are the questions I actually ask. I try to split them up and not ask every interviewer the same question.

    Questions for the recruiter

    • What is the role?
    • How many employees?
    • Type of funding? Bootstrapped? What series? (You should know if they’re public)

    Questions for engineers

    • What languages/frameworks do you use? Why does it fit your use case?
    • Where do you store code?
    • How do you deploy?
    • How is work planned?
    • How do you monitor your app?
    • Who responds to downtime? Is there a run book? Pager duty?
    • If you could change one thing at company what would it be? (A polite way to ask “What do you hate?”)

    Questions for Managers

    • How do you measure success? Six months to a year from now how do I know I’ve met my goals?
    • How does your team celebrate victories? (It’s bad if they never stop to appreciate what they’ve done)
    • What does the career path look like?
    • What does a bad sprint/project look like?

    Questions for Everybody

    • What are you looking for in a coworker/employee?

    Questions someone asked me that resulted in a good conversation but I would never ask

    • What’s your most controversial thought?
  • New Job

    As of August 14th, I have a new job!

    Oops!

    I didn’t plan to find a new job. I wanted to start my own company and was going to stay put until I could start my own thing. A friend talked me into applying for a tech lead position at his company. I had the qualifications, and it would have been a huge promotion. I applied for other jobs, but I didn’t think any of them could beat this.

    It was my first onsite interview, and I bombed it.

    Doh!

    My first onsite had only one technical interview. They asked me to model what happens when a user buys an SSL certificate. They didn’t say it explicitly, but they wanted a sequence diagram. I didn’t know what that was. I thought they wanted me to design a system, so I started mapping out services.

    My solution was way too complicated and confusing. I can blame my nerves, but I spent all of my prep-time reviewing Cracking the Coding Interview and working Leet Code’s interview questions. The question was new, so I was unprepared. Bombing this question helped me prepare for later interviews.

    I regret that my first choice was my first onsite.

    What Now?

    If I hadn’t applied to other companies, I would’ve mourned my lost free-time and given up on interviewing for another year. Instead, I doubled the number of applications I had, and continued studying in my free time.

    Job Hunt Stats

    95 days
    38 jobs applied for
    17 responses
    6 onsites
    14 rejections
    2 offers
    1 process ended once I accepted an offer
    

    Other Details

    Homework Is a Crapshoot

    Every company judges homework differently.

    I spent a week working on a homework assignment for [data metrics company] only to be rejected for doing the bare minimum. They purposefully left the scope vague to see if a candidate would go above and beyond.

    Another homework assignment I wrote in two hours and emailed it to the recruiter with notes on what I would improve if I had more time. It was good enough to skip a technical phone screen and jump straight to onsite.

    No One Thinks Twice About Your Interview

    I’d heard that interviews matter more to you than the company interviewing you. I didn’t expect to have hard proof. I ran into two people one to two weeks after they had interviewed me. Neither recognized me or remembered my interview once I brought it up.

    One exception was an interview I had with a CTO. He interviewed me when he was the VP of development at a different company. He remembered me. I should’ve asked if that was a good thing or a bad thing. 😅

    I Love My New Job

    It’s only been two weeks, but I love my new job. It’s 100% remote, and most of my team is remote. The team is brand new built to implement a killer new product. I couldn’t be more excited.

  • Who Is Your Company Afraid of Hiring?

    Does your company have crazy white boarding interviews where you’re expected to recite the curriculum of ivy league computer science programs? Your company probably has a deep existential fear of hiring someone who doesn’t know how to code.

    Does your company spend most of the post-interview aghast that the candidate said he read trashy romance novels for fun? Your company is probably most afraid of hiring a bad culture fit.

    What would it look like if your company was afraid of someone interviewing for the sole purpose of using your offer to get a raise?

    I don’t know.

    I had a weird interview experience, and this is the best explanation I have. The CTO and Director of Engineering both got really excited when I mentioned that my manager knew I was looking for a new job. I wish I had asked about it. It completely changed the tone of an interview that felt lukewarm.

  • Fixing a Postgres Type Casting Bug With Regex

    Today I learned how to fix a type casting bug in our Postgresql script with regular expressions. This deserves celebration! Details below.

    The Problem

    We have a database table that matches data to other tables based on a specific column. The values for the column are provided by the user and stored as strings. Depending on the table being matched, the values can be strings or numbers. If the user submitted bad input for a number match, accidentally including characters in their match column, the query fails.

    The code handling the input was exactly the same for every match column type. I didn’t want to add a special case for this. The problem was in the query, so the fix should be in the query.

    The Solution

    Update: I subscribe to Andy Croll’s Ruby newsletter. He suggests using \A and \Z to match the beginning and ends of a string because ^ and $ will match the ends of a line. If your string has newlines in it, it will be accepted by the regex.

    Postgres has regular expression match operators. We can update our query to check that a value is a number.

    -- Example query
    SELECT *
      FROM list_of_things lot
      WHERE lot.value ~ '^[0-9]+$' AND lot.value::BIGINT = 12002;

    The ^ operator matches the regular expression to the start of the string and the $ matches to the end. By combining them with [0-9]+ we check that every character in the string is a digit.

  • The Little Things Make Me Love Apple

    My brother was visiting and wanted to connect his iPad to our wifi. Because he’s in my contacts, I didn’t have to tell him my password; I just hit a button on my Mac.

    It’s very inconvenient for my guests to type my super long wifi password. (All of my passwords are song lyrics with random characters.) I didn’t even know Apple had this feature, and it’s made my guests’ visits smoother.

  • Apple Has to Become GE

    Apple discontinued their wireless router. They absolutely should have, but it’s a shame. The Airport Express was terrible. If you used it for a Time Machine backup, everyone else’s internet connection would stall. Unplanned automatic backups would knock my roommate off of World of Worldcraft and cause Youtube videos to stutter. It was embarrassingly bad and missed the trend of having multiple access points. Eero should never have happened.

    Peak iPhone is going to be a thing. Even if Apple was successful enough to sell an iPhone to every person on earth, they would hit peak iPhone. Investors don’t want stable, consistent returns. They want growth! Apple needs to grow its product line. It’s going to turn into G.E.

    Ignore the Homepod, which is failing because Siri is terrible. Selling more products to your current customers is the best strategy.

    Wait. What the fuck am I talking about. The Homepod might be the best product no one buys. It costs $50 more than the Bose Soundlink I bought in 2013 and has reviews comparing it to a $50,000 speaker? I… want that. Hell, I want Apple to design the power cables and other boring parts of my home appliances.

    I want to live in a world where Apple is making my toaster oven.

    1. Apple will never sell everyone a phone. People's taste are too different, and some people don't like Apple for forcing its tastes on you.
    2. I love that a failed product for Apple would be a huge success for any other company.
    3. I don't own a toaster oven. They're all terrible.
  • TicTacToe.scala Part II

    I was interviewing recently and we discussed ways to improve my TicTacToe code. Here’s version two!

    The Code

    import scala.annotation.tailrec
    import scala.util.{Try, Success, Failure}
    
    sealed trait Player
    case object X extends Player {
      override def toString: String = {
        "X"
      }
    }
    case object O extends Player {
      override def toString: String = {
        "O"
      }
    }
    
    sealed trait Status
    case class Winner(player: Player) extends Status
    case object InProgress extends Status
    case object Draw extends Status
    
    case class Tile(loc: Int, owner: Option[Player]) {
      override def toString: String = owner.map(_.toString).getOrElse(s"$loc")
    }
    
    val boardWidth = 3
    val blankState = (0 until boardWidth * boardWidth).map(loc => new Tile(loc, None))
    
    class Board(state: IndexedSeq[Tile], val currentPlayer: Player, val gameStatus: Status) {
      def move(loc: Int) : Board = {
        val (front, back) = state.splitAt(loc)
    
        val newBack = if (!back.isEmpty && back.head.owner.isEmpty) {
          val newTile = back.head.copy(owner = Some(currentPlayer))
          newTile +: back.tail
        } else {
          throw new Exception(s"Invalid location $loc")
        }
    
        val newState = front ++ newBack
        val newStatus = getNewStatus(newState, loc)
        new Board(newState, otherPlayer, newStatus)
      }
    
      private def otherPlayer : Player = {
        currentPlayer match {
          case X => O
          case O => X
        }
      }
    
      private def getNewStatus(newState: IndexedSeq[Tile], loc: Int) : Status = {
        // val (x, y) = (v % 3, v / 3)
        // 0,0 | 1,0 | 2,0
        // 0,1 | 1,1 | 2,1
        // 0,2 | 1,2 | 2,2
        val (x, y) = (loc % boardWidth, loc / boardWidth)
        val countUp = (0 until boardWidth)
        val countDown = (boardWidth - 1 to 0 by -1)
    
        def rowVictory : Boolean = countUp.forall { idx =>
          val owner = newState(idx + boardWidth * y).owner
          owner == Some(currentPlayer)
        }
    
        def columnVictory : Boolean = countUp.forall { idx =>
          val owner = newState(x + boardWidth * idx).owner
          owner == Some(currentPlayer)
        }
    
        def diagonalVictory : Boolean = {
          val isDiagonal = x == y || x + y == (boardWidth - 1)
    
          lazy val upDiagonal = countUp.forall { idx =>
            val owner = newState(idx + boardWidth * idx).owner
            owner == Some(currentPlayer)
          }
    
          lazy val downDiagonal = countUp.zip(countDown).forall { case (idx, idy) =>
            val owner = newState(idx + boardWidth * idy).owner
            owner == Some(currentPlayer)
          }
    
          isDiagonal && (upDiagonal || downDiagonal)
        }
    
        if (rowVictory || columnVictory || diagonalVictory) {
          Winner(currentPlayer)
        } else if (newState.exists(_.owner.isEmpty)) {
          InProgress
        } else {
          Draw
        }
      }
    
      override def toString : String = {
        state.grouped(boardWidth).map(_.mkString("|")).mkString("\n")
      }
    }
    
    @tailrec
    def playGame(game: Board) : Unit = {
      game.gameStatus match {
        case Winner(player) => println(s"$player is victorious!\n$game")
        case Draw => println(s"There are no moves left. It's a stupid tie.\n$game")
        case InProgress => {
          println(s"Player ${game.currentPlayer}, your move")
          println(s"$game\n")
    
          val nextMove = scala.io.StdIn.readInt()
    
          Try(game.move(nextMove)) match {
            case Success(newBoard) => playGame(newBoard)
            case Failure(ex) => {
              println(s"Error making a move\n$ex\n")
              playGame(game)
            }
          }
        }
      }
    }
    
    playGame(new Board(blankState, X, InProgress))

    Improvements

    A bunch of the improvements were insignificant. The entire diff can be viewed here.

    • Parentheses were removed from methods that didn’t take any parameters.
    • Default values for method parameters were removed to be clearer to the library user.
    • The Owner type was removed since it didn’t add any clarity.
    • The board length is now a variable, so you could easily play on an X by X game board.
    • All internal methods are now private.

    Sealed Traits

    I thought I knew about this when I first wrote TicTacToe. If the parent object of a case class is sealed, all match statements need to include every case. If every case isn’t covered, the match will throw a compile error.

    We’ll never need an instance of the parent class, so we can make the Player a trait.

    sealed trait Player
    
    sealed trait Status

    Case Objects

    The player types, X and O, don’t need to hold a value, so we should use a case object instead of a case class. Case objects are singletons, so there is only ever one of them created.

    case object X extends Player {
      override def toString: String = {
        "X"
      }
    }
    case object O extends Player {
      override def toString: String = {
        "O"
      }
    }

    Use an IndexedSeq instead of a List

    When writing a library, you want to use the most generic type possible. We use an IndexedSeq instead of a Seq because the Range type inherits from IndexedSeq. For our TicTacToe board, no extra work is done to convert our board to an IndexedSeq.

    val blankState = (0 until boardWidth * boardWidth).map(loc => new Tile(loc, None))
    
    class Board(state: IndexedSeq[Tile], val currentPlayer: Player, val gameStatus: Status) {

    Since we’re using an IndexedSeq instead of a List, we need to change how we check if a tile is empty or not. The syntax for decomposing a Seq is messy and I always forget it. We can use the isEmpty methods to check that the board space exists and that the board space is empty.

    val (front, back) = state.splitAt(loc)
    
    val newBack = if (!back.isEmpty && back.head.owner.isEmpty) {
      val newTile = back.head.copy(owner = Some(currentPlayer))
      newTile +: back.tail
    } else {
      throw new Exception(s"Invalid location $loc")
    }

    Give Better Names to Variables

    Rename nextPlayer to currentPlayer. I always confuse next Thursday and this coming Thursday. I don’t know why I thought nextPlayer was less confusing.

    Make the Game Status a Part of the Board Type

    The old TicTacToe would calculate the game status every time Board.gameState was called. This is the most expensive part of TicTacToe. We definitely want to calculate the new status only once when we create a new state for the board.

    val newState = front ++ newBack
    val newStatus = getNewStatus(newState, loc)
    new Board(newState, otherPlayer, newStatus)

    One benefit of checking for victory when the user makes a moves is we only have to check the row, column, and diagonals for that move. We can ignore the entire rest of the board. We can split these checks into three different methods. Compare the new way of checking for a victory to the old way. The logic is simpler and we get to keep our immutable variables!

    // Old way
    def findWinner(values: List[Owner]) : Status = values match {
      case List() => InProgress()
      case head :: tail => head match {
        case Some(x) => Winner(x)
        case None => findWinner(tail)
      }
    }
    
    if (movesLeft == 0) {
      Draw()
    } else {
      findWinner(placements.values.toList)
    }
    
    // New way
    if (rowVictory || columnVictory || diagonalVictory) {
      Winner(currentPlayer)
    } else if (newState.exists(_.owner.isEmpty)) {
      InProgress
    } else {
      Draw
    }

    Refactor the Game Loop to be Tail Recursive

    The TicTacToe game is recursive. We want it to be tail recursive, so it will reuse the same stack frame instead of indefinitely growing the stack. Our entire game logic is wrapped in a try catch block, but only the game.move method can throw an exception. Instead of using a lowercase try, we can use an uppercase Try, specifically a scala.util.Try. By wrapping our execution in a Try, we can assign it to a variable and use a match statement. Now our Try block has a clear execution path and always calls playGame in the tail position.

    val nextMove = scala.io.StdIn.readInt()
    
    Try(game.move(nextMove)) match {
      case Success(newBoard) => playGame(newBoard)
      case Failure(ex) => {
        println(s"Error making a move\n$ex\n")
        playGame(game)
      }
    }

    Asides

    1. Using an IndexedSeq instead of a List doesn't make a lot of sense in our example, but it was something I learned when reviewing my TicTacToe implementation with someone who knows Scala better than me. Seqs have worse syntax for decomposing them into a head and a tail than Lists do.
  • Another Rule for the Internet of Things

    Today I learned that my broken dishwasher could have shown me the error codes. I would pay extra for a dishwasher that could connect to my phone just to see those codes. I don’t want to wait a week for the repair people. I want to order the part and fix it as soon as Amazon Prime delivers.

    My first gen Automatic doesn’t reliably track my location anymore, but it’s worth keeping around for its error codes. There is no confidence like walking into a repair shop already knowing what’s wrong with your car.

    People love when appliances are repairable. If smart appliances told you what was wrong and how to fix the wrong, they’d be loved too.

  • Fun With Docker

    I really hate installing Java on my personal machines, but I still want to develop Scala. Part of the Docker promise is that you develop on the same environment as your production environment. Let’s actually set that up.

    Add this alias to your $HOME/.bashrc file. Reload your bashrc, and the sbt command will run the docker image in your current directory.

    alias sbt='docker run --rm --tty --interactive --volume "$PWD":/app bigtruedata/sbt'

    What if you want to serve html from a random directory? Create another alias for the Apache docker image! This works great for any code coverage libraries that output to html.

    alias httpd='docker run --rm --tty --interactive -p 8000:80 --volume "$PWD":/usr/local/apache2/htdocs/ httpd'