jq xq yq

tips for html, json, xml and yaml formats

jq

jq - commandline JSON processor [version 1.6]

Usage:  jq [options] <jq filter> [file...]
    jq [options] --args <jq filter> [strings...]
    jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

    $ echo '{"foo": 0}' | jq .
    {
        "foo": 0
    }

For a listing of options, use jq --help.

xq

Command-line XML and HTML beautifier and content extractor

Usage:
  xq [flags]

Flags:
  -c, --color            Force colorful output
  -e, --extract string   Extract a single node from XML
  -h, --help             Print this help message
  -m, --html             Use HTML formatter
      --indent int       Use the given number of spaces for indentation (default 2)
      --no-color         Disable colorful output
      --tab              Use tabs for indentation
  -v, --version          Print version information
  -x, --xpath string     Extract the node(s) from XML

OPML to Markdown

Turning an OPML file (eg from Overcast) into a Markdown list:

xq -r '.opml.body.outline.outline[] | ("* [" + ."@title" + "](" + ."@htmlUrl" + ")")' overcast.opml | sort -u

pretty print html output

xq -cm $FILE

yq

Usage:
  yq [flags]
  yq [command]

Examples:

# yq defaults to 'eval' command if no command is specified. See "yq eval --help" for more examples.

# read the "stuff" node from "myfile.yml"
cat myfile.yml | yq '.stuff' 

# update myfile.yml in place
yq -i '.stuff = "foo"' myfile.yml # update myfile.yml inplace


Available Commands:
  completion       Generate the autocompletion script for the specified shell
  eval             (default) Apply the expression to each document in each yaml file in sequence
  eval-all         Loads _all_ yaml documents of _all_ yaml files and runs expression once
  help             Help about any command
  shell-completion Generate completion script

Flags:
  -C, --colors                        force print with colors
  -e, --exit-status                   set exit status if there are no matches or null or false is returned
      --expression string             forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.
  -f, --front-matter string           (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
      --header-preprocess             Slurp any header comments and separators before processing expression. (default true)
  -h, --help                          help for yq
  -I, --indent int                    sets indent level for output (default 2)
  -i, --inplace                       update the file inplace of first file given.
  -p, --input-format string           [yaml|y|props|p|xml|x] parse format for input. Note that json is a subset of yaml. (default "yaml")
  -M, --no-colors                     force print with no colors
  -N, --no-doc                        Don't print document separators (---)
  -n, --null-input                    Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
  -o, --output-format string          [yaml|y|json|j|props|p|xml|x] output format type. (default "yaml")
  -P, --prettyPrint                   pretty print, shorthand for '... style = ""'
  -s, --split-exp string              print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
      --unwrapScalar                  unwrap scalar, print the value with no quotes, colors or comments (default true)
  -v, --verbose                       verbose mode
  -V, --version                       Print version information and quit
      --xml-attribute-prefix string   prefix for xml attributes (default "+")
      --xml-content-name string       name for xml content (if no attribute name is present). (default "+content")

Use "yq [command] --help" for more information about a command.
$ yq eval --front-matter=extract '.' test.md

Running the above command against the markdown for test.html:

---
title: test
subtitle: markdown test page for this blog
date: 2021-09-28
keywords: [test, markdown]

extract just the keywords:

$ yq eval --front-matter=extract '.keywords[]' test.md
test
markdown

See also