15 1 0 4000 1 https://codeblock.co.za 300 true 0
theme-sticky-logo-alt
Import CSV as Javascript Object

How to Import a CSV as a JavaScript Object

1 Comment

When building apps, especially reporting dashboards, you will often be supplied with a .csv file with raw data. Learn how to convert that data into JSON (a JavaScript object), so you can manipulate it, perform calculations and display it as you see fit.

Prerequisites

  • A basic understanding of HTML.
  • A basic understanding of JavaScript.
  • A CSV file (comma (,) separated).

A .csv file contains data that is separated by a comma, a semicolon, or some other delimiter. For this tutorial, I’ll assume your csv file uses a comma, but you can change the function to suit your needs.

Need to import a CSV file in PHP? Check my previous post here How to Import and Validate a CSV File in PHP.

Read the File

First off, to read the file, we need to use the JavaScript FileReader. The FileReader object lets web applications asynchronously read the contents of files (or raw data buffers) stored on the user’s computer, using File or Blob objects to specify the file or data to read.

We create the object and then stipulate we want to read it as plain text.

const reader = new FileReader();
reader.readAsText(file);

In the above code, the readAsText method expects the file to be supplied. This is the file that the user will upload using the HTML file input. We will build this later.

Processing the CSV File

It’s important to understand that the browser will read the file asynchronously. That means that any other functions that follow will continue to run even if your file is not read yet. This will be a problem if you have a large CSV.

To avoid this, we can create a promise, and only once that promise is resolved, we’ll do something with the data.

The Promise

Creating a promise is simple task as follows:

const reader = new FileReader();
reader.readAsText(file);
const promise = new Promise( (resolve, reject) => {
      // Do something here
})

The promise will either resolve (success) or reject (fail) and we stipulate what determines a success or failure.

Let’s add the basic FileReader onload and onerror methods into the promise, then setup the conditions responsible for the resolve and reject.

const promise = new Promise( (resolve, reject) => {

      // If the file reader doesn't load, we'll reject the promise
      reader.onerror = (event) => reject("Failed to read file");

      // If the file reader loads, we'll process the data, but may still reject the promise if other conditions are not met
      reader.onload = (event) => {
            //    Just because the file reader has loaded, doesn't mean everything is fine
            //    We can set up other conditions here to reject the promise
            //    For example, your csv may require a fixed number of columns
            //    If the uploaded CSV has a different number, we can reject the promise

            // If we encounter errors along the way, push them to this errors array and notify the user later.
            let errors = [];

            // This array will contain our final data
            const objArray = [];

            if(errors.length === 0) {
                  return resolve(objArray );
            }
            return reject(errors);
            
      }
      
});

In the above code, I’ve gone a little further to set up an empty array we’ll use to collect errors. If the uploaded CSV file doesn’t meet certain criteria, we can reject the promise and notify the user of each one of them on the page.

I’ve also created an empty “objArray” array we will use to store the objects we create from the CSV data.

Now we move on to the good part.

Convert CSV Rows to JavaScript Object

If the reader manages to load, we can read the plain text data in the CSV via the event target’s result. To ensure we aren’t getting any invalid rows, it’s best to trim() the entire string, removing any whitespace from the beginning and end.

...

// This object will contain our final object
const array = [];

const csvText = event.target.result.trim();

...

The data I’m using looks like this in Microsoft Excel.

If we were to console.log the above, we would get something that looks like the below.

...

const csvText = event.target.result.trim();
console.log(csvText);

// "First Name,Last Name,Alias\r\nTony,Stark,Ironman\r\nSteve,Rogers,Captain America\r\nNatasha,Romanov,Black Widow\r\nBruce,Banner,Hulk"
...

Notice the \r and \n. These are returns and new lines. They basically signify the end of the row and the start of the new row. This is important as it will help us segment that data and ensure that the columns are being grouped correctly.

Split the Data into Separate Rows

Next we need to create lines using those returns and new lines. Here we will the built-in JavaScript split() function.

...

console.log(csvText);
const rows = csvText.split(/\r\n/g);
console.log(rows);

...

Now we have an array of strings, each array value like so:

[
      "First Name,Last Name,Alias",
      "Tony,Stark,Ironman",
      "Steve,Rogers,Captain America",
      "Natasha,Romanov,Black Widow",
      "Bruce,Banner,Hulk"
]

Format the CSV Headers to Use as JavaScript Object Keys

Since our CSV has headers, we want to use those headers as keys in each of our object. We want to make it a reference so we can reuse it for all the rows and also want to remove it from the original array.

We can use JavaScript’s array.shift() method which will handle both these tasks for us.

While I do that, I am also going to split the row, using the comma as a separator this time. This will create an array with each column header as a value.

const columnHeaders = rows.shift().split(",");
          
console.log(columnHeaders);

// columnHeaders = ['First Name', 'Last Name', 'Alias']

In the above code, the first line does the following:

  • Remove the first value from the array.
  • Take that value which was removed (in our case, the first string) and split it by the commas, creating a new array of items.
  • Finally, save this new data into our columnHeaders constant.

All that’s left to do now is push the rest of the rows to the objArray we created earlier, but first we need to convert each row into an object using the headers as keys.

Convert Each CSV Row into a JavaScript Object and Push to the Array of Objects

Similar to how we created the headers, we will loop through each of the strings that are left in the array, split them, then push them.

rows.forEach( (row, rowIndex) => {
    const columns = row.split(",");
    const obj = {};

    for(let i = 0; i < columHeaders.length; i++) {
        if(columns[i] === undefined) {
            errors.push(`Column ${i} on row ${rowIndex} is missing.`);
        }
        // else if(columHeaders="Account Number" && isNaN(columns[i]) ) {
        //     errors.push(`Column ${i} on row ${rowIndex} is not a valid number.`);
        // }
        else {
            obj[columHeaders[i]] = columns[i];
            
        }
    }
    objArray.push(obj);
})

In the above code, we loop through the remain items and perform the following tasks.

  • Split the text by the comma to create an array of values.
  • Create an empty object we will use to store our key:value pairs as we loop through each each value in our columns.
  • Use a for loop to loop through the column headers we created earlier, using the index of the header to identify the index of the column associated with it. You may instead choose to loop through the column, however it’s safer to loop through the headers since every column requires a header.
  • Add a key:value pair to the object (header:column).
  • Finally, push the object to the objArray.

Error Checking

In the comments, of the above code, you’ll notice I’m doing some validation. This is where that errors array we created earlier comes in handy. So we can still reject the promise based on the data types or pretty much any other validation we want.

The Javascript Array of Objects

If we console.log the objArray now, we will get this, based on my data.

[
    0 :  {First Name: 'Tony', Last Name: 'Stark', Alias: 'Ironman'}
    1 :  {First Name: 'Steve', Last Name: 'Rogers', Alias: 'Captain America'} 2 :  {First Name: 'Natasha', Last Name: 'Romanov', Alias: 'Black Widow'} 3 :  {First Name: 'Bruce', Last Name: 'Banner', Alias: 'Hulk'}
]

Now I can get a first name by going objArray[0][“First Name”].

How to Deal with Commas in CSV Column Data

Sometimes, especially when dealing with addresses or user submitted text, you may find that CSVs contain commas within text that is within quotations. Something like this:

hello, world, "street, suburb, city, country"

This is a problem.

If you try to split this with the comma, you will end up with the below, which is a nightmare for data integrity.

[
    hello, 
    world, 
    "street, 
    suburb, 
    city, 
    country"
]

Instead of a plain comma to separate the headers and columns, let’s switch to a regex pattern. We’ll create a new function to handle this part as it will make it easier to read later.

const splitCSVColumns = (string) => {
    const array = string.split(/,(?=(?:(?:[^"]*"){2})*[^"]*$)/);
    return array.map(value => value.trim().replace(/^"|"$/g, ""));
}

The above function does two things.

  1. First, it splits the rows by columns using a comma, but only if that comma does appear between quotations.
  2. The second part remove quotations that were added when the CSV was created. Quotations in a CSV file typically indicate that there is text that is separated by spaces.

The first regex which is responsible for the splitting, works like this:

  • , is the character we want to match.
  • (?=…) is a positive lookahead, which means “is followed by”. It checks for a certain pattern without including it in the match.
  • (?:[^”]“){2} matches a pair of double quotes with anything in between. [^”] matches any number of characters that are not double quotes, and {2} specifies that we want two of these (i.e., a pair of double quotes).
  • (?:(?:[^”]“){2}) matches any number of these pairs of double quotes.
  • [^”]*$ matches any number of characters that are not double quotes, up until the end of the string.

So, in simple terms, this regex matches a comma only if it is followed by an even number of double quotes (which means it’s outside of double quotes) until the end of the string.

Create a Reusable Function to Convert CSV Data to JavaScript Object

Let’s make this reusable by creating a function we can call the time we need it, and the function we’ll use as the “string splitter”.

const splitCSVColumns = (string) => {
    const array = string.split(/,(?=(?:(?:[^"]*"){2})*[^"]*$)/);
    return array.map(value => value.trim().replace(/^"|"$/g, ""));
}

const readCSV = (file) => {
    const reader = new FileReader();
    reader.readAsText(file);
    const promise = new Promise((resolve, reject) => {

        // If the file reader doesn't load, we'll reject the promise
        reader.onerror = (event) => reject("Failed to read file");

        // If the file reader loads, we'll process the data, but may still reject the promise if other conditions are not met
        reader.onload = (event) => {
            //    Just because the file reader has loaded, doesn't mean everything is fine
            //    We can set up other conditions here to reject the promise
            //    For example, your csv may require a fixed number of columns
            //    If the uploaded CSV has a different number, we can reject the promise

            // If we encounter errors along the way, push them to this errors array and notify the user later.
            let errors = [];

            // This array will contain our final data
            const objArray  = [];

            const csvText = event.target.result.trim();

            const rows = csvText.split(/\r\n/g);

            const columHeaders = splitCSVColumns(rows.shift());
            
            rows.forEach( (row, rowIndex) => {
                const columns = splitCSVColumns(row);
                const obj = {};

                for(let i = 0; i < columHeaders.length; i++) {
                    if(columns[i] === undefined) {
                        errors.push(`Column ${i} on row ${rowIndex} is missing.`);
                    }
                    // else if(columHeaders="Account Number" && isNaN(columns[i]) ) {
                    //     errors.push(`Column ${i} on row ${rowIndex} is not a valid number.`);
                    // }
                    else {
                        obj[columHeaders[i]] = columns[i];
                        
                    }
                }
                objArray.push(obj);
            })

            if (errors.length === 0) {
                return resolve(objArray );
            }
            return reject(errors);

        }

    });

    return promise;
}

Using the CSV to JavaScript Function

Now, to use this, we simply have to supply the file to the function, then when the promise resolves, we can do something with the data.

In the below, I’ve created a simple HTML input we’ll use to upload a CSV file. When there is a change on the document, we’ll call our function, supplying the file then simply show the object on the page as plain text.

Remember, we created a promise, so we need to use the .then(), .catch() and optionally, the .finally() methods to use the data.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
</head>

<body>

    <input id="fileInput" type="file" accept=".csv" />

    <div id="preview" style="margin-top: 20px"></div>

    <script>
        const fileInput = document.getElementById("fileInput");

        const splitCSVColumns = (string) => {
            const array = string.split(/,(?=(?:(?:[^"]*"){2})*[^"]*$)/);
            return array.map(value => value.trim().replace(/^"|"$/g, ""));
        }

        const readCSV = (file) => {
            const reader = new FileReader();
            reader.readAsText(file);
            const promise = new Promise((resolve, reject) => {

                // If the file reader doesn't load, we'll reject the promise
                reader.onerror = (event) => reject(["Failed to read file"]);

                // If the file reader loads, we'll process the data, but may still reject the promise if other conditions are not met
                reader.onload = (event) => {
                    //    Just because the file reader has loaded, doesn't mean everything is fine
                    //    We can set up other conditions here to reject the promise
                    //    For example, your csv may require a fixed number of columns
                    //    If the uploaded CSV has a different number, we can reject the promise

                    // If we encounter errors along the way, push them to this errors array and notify the user later.
                    let errors = [];

                    // This array will contain our final data
                    const objArray  = [];

                    const csvText = event.target.result.trim();

                    const rows = csvText.split(/\r\n/g);

                    const columHeaders = splitCSVColumns(rows.shift());
                    
                    rows.forEach( (row, rowIndex) => {
                        const columns = splitCSVColumns(row);
                        const obj = {};

                        for(let i = 0; i < columHeaders.length; i++) {
                            if(columns[i] === undefined) {
                                errors.push(`Column ${i} on row ${rowIndex} is missing.`);
                            }
                            // else if(columHeaders="Account Number" && isNaN(columns[i]) ) {
                            //     errors.push(`Column ${i} on row ${rowIndex} is not a valid number.`);
                            // }
                            else {
                                obj[columHeaders[i]] = columns[i];
                                
                            }
                        }
                        objArray.push(obj);
                    })

                    if (errors.length === 0) {
                        return resolve(objArray );
                    }
                    return reject(errors);

                }

            });

            return promise;
        }

        fileInput.addEventListener("change", (e) => {
            if(e.target.files.length > 0 && e.target.files[0] && e.target.files[0].type === "text/csv") {
                readCSV(e.target.files[0])
                .then( (csvRows) => {
                    const preview = document.getElementById("preview");
                    preview.innerHTML=JSON.stringify(csvRows)
                    .replaceAll("{", "&nbsp;&nbsp;&nbsp;&nbsp;{")
                    .replaceAll("[", "[<br>")
                    .replaceAll("]", "<br>]")
                    .replaceAll("},", "},<br>");
                })
                .catch( (errors) => {
                    console.log(errors);
                })
                .finally( () => {
                    // You can do something here, like hide a loader or enable a disabled button.
                })
            }
        })
    </script>
</body>

</html>

Get it on GitHub

If you would like to use the code in your project, feel free to copy and paste the functions from above or grab the final code from GitHub.

Thanks for reading.

Code References

JavaScript

Is this still valid in 2024? Please let me know in the comments below.

Was This Helpful?

How to Import a CSV File in PHP
Previous Post
How to Import and Validate a CSV File in PHP
How to Speed Up WordPress Websites
Next Post
How to Speed Up Your WordPress Website

1 Comment

Leave a Reply