Sync Cookies With cURL

I wanted to reuse authenticated sessions from the browser to call protected APIs with cURL (curlie to be precise). Fortunately cURL can use Netscape HTTP Cookie File as a cookie jar. There are extensions for Firefox that can export cookies in this format. Unfortunately they require too much manual clicking. We can do better, right? Well… I came up with semi-automatic process.

Extracting cookies from Firefox

FF keeps cookies in SQLite database. There is a script you can use to convert this database to Netscape format.

Session cookies

The script is not able to extract session cookies since they are not stored in the database. Luckily Firefox stores the content of session cookies in sessionstore-backups/recovery.baklz4 file. The file is basically json in Firefox’s weird lz4 envelope. lz4cat is not able to read it - you will need lz4jsoncat utility (lz4json package on Debian based systems).

Conversion from recovery.baklz4 to Netscape format is trivial with jq

lz4jsoncat "$SESSION_FILE" |
	jq --raw-output '
        .cookies[] |
        (.host | if test("^[.]") then "TRUE" else "FALSE" end) as $flag |
        (.secure | tostring | ascii_upcase) as $secure |
        "\(.host)\t\($flag)\t\(.path)\t\($secure)\t0\t\(.name)\t\(.value)"
    '

The forked hackerb9’s script with session cookies extraction is available as gist.

Fair warning tho: it seems that sometimes Firefox is not flushing cookies to disk immediately. You can try clicking extension’s icon. It seems that it forces cookies to be stored on disk. I haven’t found any workaround for this issue yet.

curlrc

You can configure cURL to use the Netscape cookies file as cookie jar. cURL reads configuration from ~/.curlrc file:

# Save cokies to file
cookie-jar <PATH TO COOKIES FILE>
# Attach cookies to sent requests
cookie <PATH TO COOKIES FILE>

More info about curlrc is available on curl’s manpage.