Extract the full Routes instead of Top Routes using ipinfo summarize (CLI)

[aalshaya]

Hello, I have a question regarding the ‘ipinfo summarize’ CLI command. Whenever I execute this command, one of the outputs I receive is ‘Top Routes’, which compiles all the input IPs into subnets. However, my problem is when I input a large number of IPs: it only displays the top 5 or 6 subnets. Is there a method to show full Routes instead of Top Routes

[Abdullah]

Hey aalshaya,

I really appreciate you posting this question in our community! Thank you very much. Let’s break down the problem.

Our list of IP addresses

Let’s say you have the ips.txt file containing 1,000 IP addresses. Please note that these IP addresses are generated randomly and may contain bogon IP addresses. You can generate random IP addresses with using this website: Random IP Generator

$ wc -l ips.txt
1000 ips.txt

Exploring the summarize command

To summarize the IPs in the ips.txt file you can run the following command:

ipinfo summarize ips.txt

However, it only yields the top results and limits the results to 5.

$ ipinfo_cli_duckdb_summarize $ ipinfo summarize ips.txt

Summary
- Total   1000
- Unique  1000
- Anycast 0
- Bogon   139
- Mobile  41
- VPN     10
- Proxy   0
- Hosting 135
- Tor     0
- Relay   0

[...]

Top Routes
- 30.0.0.0/8 (AS749) 7 (0.7%)
- 55.0.0.0/8 (AS721) 6 (0.6%)
- 26.0.0.0/8 (AS749) 6 (0.6%)
- 22.0.0.0/8 (AS749) 6 (0.6%)
- 29.0.0.0/8 (AS749) 6 (0.6%)

[...]

Bulk enrichment

To obtain all available information from the IP addresses, you need to use the bulk command on the API. The bulk command will essentially perform bulk IP metadata enrichment on the IP addresses you provide. Considering you have the proper API access (see the pricing page), you can run the following command for the bulk IP enrichment:

ipinfo bulk -c ips.txt > ips_ipinfo.csv

Here, I am using the -c parameter to get the CSV output after the bulk enrichment, and I am storing the enriched IP address data in the ips_ipinfo.csv file.

After running the command ips_ipinfo.csv file will be generated. The ips_ipinfo.csv file will look something like this:

Extracting the top routes from the CSV file

You can use whatever dataframe or database libraries you want here. I am using duckdb because it is convenient. There are a ton of solutions out there. After installing duckdb, I will get the top routes using a simple SQL query. Here is my code:

SELECT Asn_route, count(*) as route_count  FROM "./ips_ipinfo.csv"
WHERE Asn_route is not null
GROUP BY Asn_route
ORDER by route_count desc

Asn_route → The column contains the route information.

Output:

And that’s it. Please let me know if you have any follow queries. Thanks!

[Abdullah]

Please note that the IP metadata information you will receive after bulk enrichment will depend on the API tier you are on. The ASN level information is available on the basic tier API. You can use the free IP to ASN database to receive some IP information, though that technique is different. Let me know if you have any questions.

[aalshaya]

Hello Abdullah, I’m truly grateful for your efforts in this matter. Yet, I’ve encountered an issue: when I run the bulk command for random IPs, the ‘Asn_route’ field doesn’t appear. Do you have any insights into what might be causing this problem?

[aalshaya]

Alright, I believe that subscribing to the Basic plan should provide me with this information, correct?

[Abdullah]

Yes, the ASN data is not available on the free tier API access. You need to subscribe to the basic tier. With the basic tier, you will also get access to a higher number of requests that you will need because bulk enrichment means that you need to make an API request for each IP address.

If you want a free solution, you can try out a somewhat complicated solution with our free IP to ASN database.

For this you need to start by compiling the CSV file format of the free IP to ASN database using our mmdbctl tool:

# basic CSV importing into MMDB.
$ mmdbctl import --in data.csv --out data.mmdb

Then go through the following Python code ↓

Code to get ASN routes from the free IP to ASN database (github.com)

This is a complicated process as you need to regularly download the IP to ASN database, compile the CSV to MMDB, and create scripts based on the Python script that I wrote for you. But the benefit is that with a database, you can make unlimited queries and it is free.

But, the basic tier API is convenient and does come with a lot ASN related information. It is your call Let me know what you think.

[aalshaya]

This is a really an amazing job; I’ll check out the free plan first and see how it goes. If I’m stuck , I’ll definitely consider subscribing.

[Abdullah]

Awesome. My pleasure. Feel free to explore the community, we have a ton of resources here