- Three Ways to Retrieve JSON from the Web using Node.js
- JSONView, a super-handy web extension for Chrome and Firefox that pretty-prints JSON in your browser
- Replicating fetch() with ‘node-fetch’ package
- Using the http/https modules provided by Node.js
- Simplifying syntax with ‘request’ package
- Conclusion
- JavaScript read JSON from URL
- URL
- JSON
- Reading JSON with JQuery
- Reading JSON with Fetch API
- Reading JSON with XMLHttpRequest
- Author
- How to read a JSON file from a URL in JavaScript
- What is JSON?
- Read JSON file from URL
Three Ways to Retrieve JSON from the Web using Node.js
Cover image credit: Hunter x Hunter manga by Yoshihiro Togashi, meme-ified by yours truly. In a recent technical challenge, I was asked to build a small Node.js app that first needed to retrieve some JSON from the web. Since I’m still relatively new to Node.js, I did not realize that Node.js does not natively include the fetch() API, which I was used to using in my front-end JavaScript. (What I didn’t realize is that fetch() is actually a method from the Window interface—and not having a front-end, there was no window! Awkward thing to realize at the beginning of a technical interview. ) After getting through the challenge, I spent this past weekend refactoring the code and experimenting with a few different ways to retrieve JSON from the web using Node.js. Spoiler: there’s an http/https module in Node.js, but also some cool packages that mimic fetch(), or that simplify request syntax even further! But before we get into that, I want to introduce a tool that I foolishly did not use during my technical challenge: JSONView, a super-handy web extension for Chrome and Firefox that pretty-prints JSON in your browser.
JSONView, a super-handy web extension for Chrome and Firefox that pretty-prints JSON in your browser
One issue I had during my code challenge was (due to my own error) not working on my own computer, which includes this extension. Compare the following:
Unformatted JSON from https://www.reddit.com/r/popular.json versus
Same JSON from https://www.reddit.com/r/popular.json, pretty-printed with JSONView On top of that, hovering your cursor over a particular field will show the path to access it:
Cursor hovering over the «ups» field, with the path shown in the bottom-left corner Having this handy will make parsing and accessing the data you need way faster and easier.
Replicating fetch() with ‘node-fetch’ package
The node-fetch package does pretty much what you expect: provide you with the fetch() syntax in Node.js. To install, run npm install node-fetch , and set up your code like this:
const fetch = require('node-fetch'); let url = "https://www.reddit.com/r/popular.json"; let settings = method: "Get" >; fetch(url, settings) .then(res => res.json()) .then((json) => // do something with JSON >);
Here, we’ve started by importing the package via require() , and created a settings variable to define our http method as a Get request. From there, we use fetch(url, settings) just like we would on the front-end. As usual, we can parse the response res as JSON, and then do whatever we need to with it. Note: from some VERY RUDIMENTARY benchmark testing, it appears that node-fetch is the fastest of the three options covered in this article. Here are the times clocked by each (however, this DOES include running the rest of the code from the challenge, not just the fetch/https/request itself):
fetch: 0.689 seconds https: 2.827 seconds request: 3.65 seconds
I’d love for someone else to do a little more testing and verify/disprove this! Feel free to comment below if you’re that person. 😉
Using the http/https modules provided by Node.js
Node.js comes with a pair of http/https modules, and in this case, the https module provides a built-in method for Get requests. Here’s the code we’ll be looking at:
const https = require('https'); let url = "https://www.reddit.com/r/popular.json"; https.get(url,(res) => let body = ""; res.on("data", (chunk) => body += chunk; >); res.on("end", () => try let json = JSON.parse(body); // do something with JSON > catch (error) console.error(error.message); >; >); >).on("error", (error) => console.error(error.message); >);
There’s a bit more going on here! First, we import the https module with require() . We can then call https.get(url, (res) => <> ) to initiate a Get request. Then, inside the body of the callback, we start by creating an empty string body that we’ll add our the text of our response (again called res ) to. From there, we have a few examples of the .on syntax, which will listen for a few different events—namely, «data» , «end» , and «error» . When the response encounters «data» , we add each chunk as text to our body variable. Once we hit the «end» of the response, we use the try / catch syntax to try to parse our body’s text as JSON, and return an error if it can’t. Lastly, we chain another .on call to catch «error» for our initial https.get() request. I find this syntax to be pretty clunky and verbose, although I do like the explicit error handling that is required by https.get() . However, this module is slower than the node-fetch package—see the benchmark results above.
Simplifying syntax with ‘request’ package
The third strategy I used was the request package, which aims to simplify the (often verbose) syntax of Node.js’s http requests. Since this is an external package, start by installing it with npm install request . Here’s the code we’ll be looking at:
const request = require('request'); let url = "https://www.reddit.com/r/popular.json"; let options = json: true>; request(url, options, (error, res, body) => if (error) return console.log(error) >; if (!error && res.statusCode == 200) // do something with JSON, using the 'body' variable >; >);
Wow, that’s really readable! Let’s break it down. As with the other examples, we import the package with require() , and set our url variable. The request package also has a nifty options feature, where you can specify a lot of things—but here, in setting < json: true >, we tell the request to automatically parse the response’s body as JSON if there’s no error (and we get a 200 status code back). So, to access the JSON we want, just use the body variable! This readability comes at the price of speed, however. Per the benchmark results above, this is the slowest option, most likely because so much is happening under-the-hood. However, the readability is top-notch, and configuring other http requests are just as simple as this Get request example!
Conclusion
This particular technical challenge was a great opportunity to dive into Node.js’s http requests! Now, you should feel armed with a variety of tools to meet different situations. As I said above, I’d love to have another person do some testing/benchmarking and verify or disprove the speed test results that I got! Since testing is still relatively new to me, I’d very much like to see how others approach benchmarking these methods. Thanks for reading, and feel free to comment below!
JavaScript read JSON from URL
JavaScript read JSON from URL tutorial shows how to read data in JSON format from the provided URL. We use JQuery, Fetch API, and XMLHttpRequest.
URL
A , is a reference to a web resource that specifies its location on a computer network and a mechanism for retrieving it. A web resource is any data that can be obtained via web, such as HTML documents, PDF files, PNG images, JSON data, or plain text.
A generic URL has the following form:
scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]
The square brackets indicate that the part is optional. A scheme is a way of addressing resources, such as http, ftp, mailto, or file.
The part following two slashes is called the authority part. The authority part contains 1) an optional authentication section of a user name and password, separated by a colon, followed by an at symbol (@) 2) a host, which is either a host name of or an IP address, 3) an optional port number, separated from the host by a colon.
A path is a road to the resource on the host. It may or may not resemble or map exactly to a file system path. Query string is used to add some criteria to the request for the resource. It is often a sequence of key/value pairs. The final part is an optional fragment, which points to a secondary resource, such as a heading. It is separated from the query string by a hash (#).
JSON
is a lightweight data-interchange format. It is easy for humans to read and write and for machines to parse and generate. The official Internet media type for JSON is application/json . The JSON filename extension is .json .
In our examples, we use JSON data from http://time.jsontest.com .
The GET request returns this JSON string.
Reading JSON with JQuery
is a JavaScript library which is used to manipulate DOM. With jQuery, we can find, select, traverse, and manipulate parts of a HTML document.
The JQuery $.getJSON method loads JSON-encoded data from a server using a GET HTTP request.
jQuery.getJSON( url [, data ] [, success ] )
This is the method signature. The url parameter is a string containing the URL to which the request is sent. The data is a plain object or string that is sent to the server with the request. The success is a callback function that is executed if the request succeeds.
$.getJSON is a shorthand for the above call.
$.getJSON('http://time.jsontest.com', function(data) < var text = `Date: $
Time: $
Unix time: $` $(".mypanel").html(text); >);
In the example, we read JSON data from http://time.jsontest.com . The returned object has three attributes: date, time, and unix epoch.
var text = `Date: $
Time: $
Unix time: $`
We build the message using the JavaScript template string.
With JQuery’s html method, we append the text to the div tag. Figure: Reading JSON from URL with JQuery
In the figure we can see the current date, time, and Unix time.
Reading JSON with Fetch API
is interface for fetching resources. It is similar to XMLHttpRequest but its API provides a more powerful and flexible feature set.
The example reads JSON data with Fetch API and prints the returned data to the console. To see the output, we need to activate the developer console of our browser.
The fetch method takes one mandatory argument, the path to the resource we want to fetch. It returns a promise that resolves to the response of the request.
Reading JSON with XMLHttpRequest
API provides client functionality for transferring data between a client and a server. It allows an easy way to retrieve data from a URL without having to do a full page refresh. As a consequence, a web page has to update just a part of the page without disrupting what the user is doing. XMLHttpRequest is used heavily in AJAX programming.
This example reads JSON data with XMLHttpRequest .
var xhr = new XMLHttpRequest();
A new instance of XMLHttpRequest is created.
The open method initializes a request.
The responseType value defines the response type.
In the onload method, we wait for the response from the server.
The send method sends the request; the request is asynchronous by default.
In this article we have read JSON data in JavaScript with JQuery, Fetch API, and XMLHttpRequest.
Author
My name is Jan Bodnar and I am a passionate programmer with many years of programming experience. I have been writing programming articles since 2007. So far, I have written over 1400 articles and 8 e-books. I have over eight years of experience in teaching programming.
How to read a JSON file from a URL in JavaScript
Many candidates are rejected or down-leveled due to poor performance in their System Design Interview. Stand out in System Design Interviews and get hired in 2023 with this popular free course.
What is JSON?
JSON (JavaScript Object Notation) is the standard design for human-readable data interchange. It is a light-weight, human-readable format used for storing and transporting data. Since JSON works with a tree structure, it looks like an XML Extensible Markup Language (XML) a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable. . For developers, XML is shorter to write and very easy to comprehend due to its many built-in features. XML is mostly used when data is sent from a server to a web page.
Read JSON file from URL
There are several online websites that contain JSON data in order to test and apply their algorithms. I will test one such algorithm on JSON data. In JavaScript, there are several methods to read JSON data through a URL such as jqeury, loadJSON methods, etc. In this shot, we’ll look at a simple loadJSON method for better understanding.
- Click here to view the JSON data that you need to load. This JSON data contains information about 100 different posts (as can be seen below). There are 4 different fields namely userId , id , title , and body . The id field represents the post number.