IPinfo CLI bulk command: Filter Bogon IP addresses with JQ

[Abdullah]

JQ is a command line tool for processing and manipulating JSON data.

jq

JQ is one of the tools that complement our CLI really well. With JQ you can extend our CLI’s abilities.

bulk command crash course

The CLI’s bulk command can run bulk IP lookup operations from a list of IP addresses. You can pass an IP range, a CIDR, and a list of IPs and pipe IP addresses as input. The command supports both CSV and JSON output.

ipinfo bulk ips.txt

Manual

Usage: ipinfo bulk [<opts>] <ip | ip-range | cidr | filepath>

Description:
  Accepts IPs, IP ranges, CIDRs and file paths.

Examples:
  # Lookup all IPs from stdin ('-' can be implied).
  $ ipinfo prips 8.8.8.0/24 | ipinfo bulk
  $ ipinfo prips 8.8.8.0/24 | ipinfo bulk -

  # Lookup all IPs in 2 files.
  $ ipinfo bulk /path/to/iplist1.txt /path/to/iplist2.txt

  # Lookup all IPs from CIDR.
  $ ipinfo bulk 8.8.8.0/24

  # Lookup all IPs from multiple CIDRs.
  $ ipinfo bulk 8.8.8.0/24 8.8.2.0/24 8.8.1.0/24

  # Lookup all IPs in an IP range.
  $ ipinfo bulk 8.8.8.0-8.8.8.255

  # Lookup all IPs from multiple sources simultaneously.
  $ ipinfo bulk 8.8.8.0-8.8.8.255 1.1.1.0/30 123.123.123.123 ips.txt

Options:
  General:
    --token <tok>, -t <tok>
      use <tok> as API token.
    --nocache
      do not use the cache.
    --help, -h
      show help.

  Outputs:
    --field <field>, -f <field>
      lookup only specific fields in the output.
      field names correspond to JSON keys, e.g. 'hostname' or 'company.type'.
      multiple field names must be separated by commas.
    --nocolor
      disable colored output.

  Formats:
    --json, -j
      output JSON format. (default)
    --csv, -c
      output CSV format.

Why you might need to do Bogon filtering.

Some IP addresses and IP ranges are reserved for special use, such as for local or private networks, and should not appear on the public internet. These reserved ranges, along with other IP ranges that haven’t yet been allocated and shouldn’t appear on the public internet, are sometimes known as bogons.

Source: Bogon IP Address Ranges - IPinfo.io

If you are running IP log data enrichment or running our grepip command, you will come across bogon IP addresses that do not have any IP data information associated with them. They can sometimes be distracting, so we might need to filter them out.

Filtering out bogon IP addresses from the CLI bulk command

My IP address list:

ips_email.txt

023.07.10.04
2002:a0d:cad6:0:b0:576:94e1:efec
2002:a0d:cc8e:0:b0:57a:2250:f5c0
2002:a5d:6047:0:0:0:0:0
209.85.220.73
::

Command to filter out bogon IPs:

ipinfo bulk ips_email.txt | jq 'to_entries | map(select(.value.bogon != true)) | from_entries'

Output:

JQ function breakdown:

• to_entries convert the input to a key-value like pairs
• map(select(.value.bogon != true)) Uses a combination of map and select command to find out which element does not have the bogon field as not true aka false.
• form_entries convert the key-value like pairs to the original JSON object.