Bash and quoted strings with Elasticsearch

This is one of those fun edge cases where you don't know that you don't know something until it beats you over the head for about an hour.

My situation was simple - I was trying to add a document to Elasticsearch via curl. I've done it multiple times in the past without issue, yet this snippet in particular wouldn't work:

curl -XPOST 'localhost:9200/trivia/jeopardy/?pretty' -d '{
  "category": "HISTORY",
  "air-date": "2004-12-31",
  "question": "'For the last 8 years of his life, Galileo was under house arrest for espousing this man's theory'",
  "value": "$200",
  "answer": "Copernicus",
  "round": "Jeopardy!",
  "show_number": "4680"
}'

I was consistently seeing bizarre errors such as the console expecting an additional " at the end. Doing so of course resulted in an error. Different variations of the command resulted in other errors such as a mapper_parsing_exception when the inner single quotes of question were removed. This kept up for some time.

Eventually I decided to make the snippet as simple as possible and stripped out anything special. After all, I've previously run numerous commands in bash without issue... including curl commands from the book Elasticsearch In Action (great read, by the by). Turns out the curl command works fine when stripped down to:

curl -XPUT 'localhost:9200/trivia/jeopardy/?pretty' -d '{  
  "category": "HISTORY",
  "air-date": "2004-12-31",
  "question": "For the last 8 years of his life Galileo was under house arrest for espousing this mans theory",
  "value": "200",
  "answer": "Copernicus",
  "round": "Jeopardy",
  "show_number": "4680"
}'

With this tidbit of information in hand, it became apparent that something was amiss. After some reflection, I realized it could be the console itself and I happened to be using bash. It turns out bash handles the intermingling of single quotes, double quotes, and special characters (e.g., !, $) in an unexpected manner. I was able to confirm this is a pain point for at least one other person. The comments on that page are quite informative.

As I was not satisfied with the potential workarounds within bash, I decided to use a REST client add-on for Firefox instead. Certainly this is not a very satisfying solution, but since the command is only going to be used for some basic testing purposes it is simply not worth the time or pain.

Photo by Philip Estrada / Unsplash