Data Set

This module provides a way to import a comma-separted values file (CSV) and display the data as an HTML table, or in other ways provided by the developer.

To import a CSV you can

1. Open the Manager Modal
2. Import a CSV file
3. Edit and Publish the Data Set

To render the Data Set on the page, you can

4. Edit the page by adding the Data Set Widget to the page
5. Edit the Data Set Widget options
6. Render the Data Set Widget

Installation

To install the module, use the command line to run this command in an Apostrophe project's root directory:

npm install @apostrophecms-pro/data-set

Usage

Configure the Data Set module in the app.js file:

const apostrophe = require('apostrophe');

apostrophe({
  shortName: 'my-project',
  modules: {
    '@apostrophecms-pro/data-set': {},
    '@apostrophecms-pro/data-set-widget': {}
  }
});

Parsers

You can create your own parser for formats other than CSV.

Let's create a csv parser (note: csv parser is provided by default in defaultParsers, it is shown here as an example).

// modules/@apostrophecms-pro/data-set/index.js
const parserCsv = require('./lib/parsers/csv.js');

module.exports = {
  options: {
    parsers: {
      csv: parserCsv
    }
  }
};

A parser need to expose the following properties

• name: the parser name
• label: the parser label, like the description
• allowedExtensions: use as the accept attribute for the file field (please refer to MDN Limiting accepted file types
• parse: asynchronous function use to return the data and the columns. Receives { filePath, reporting } and returns { data, columns }
• getTotal: asynchronous function used to return the total number of records for the Data Set. Receives filePath and returns an integer

const fs = require('fs');
const readline = require('readline');
const { parse: csvParse } = require('csv-parse');

module.exports = {
  name: 'csv',
  label: 'CSV (comma-separated values)',
  allowedExtensions: '.csv',
  async parse({
    filePath,
    reporting
  }) {
    const piece = {
      columns: [],
      data: []
    };

    const parser = fs
      .createReadStream(filePath)
      .pipe(csvParse({
        columns: headers => {
          piece.columns = headers;

          return headers;
        }
      }));

    for await (const record of parser) {
      piece.data.push(record);
      reporting.success();
    }

    return piece;
  },
  async getTotal(filePath) {
    return new Promise((resolve, reject) => {
      let linesCount = -1;
      const rl = readline.createInterface({
        input: fs.createReadStream(filePath),
        output: process.stdout,
        terminal: false
      });
      rl.on('line', (line) => {
        linesCount++;
      });
      rl.on('close', () => {
        resolve(linesCount + 1);
      });
    });
  }
};

parse

It receives an object with

• filePath: the file path of the uploaded file
• reporting: a function that exposes a success method that you can call to update the frontend progress bar. If getTotal returns 10, you can call reporting.success() 10 times to reach 100%. You can also call reporting.end() if you want to complete the progress bar immediately.

It returns an object with

• data: an array of objects
• columns: an array containing the keys from the data property. It will be used as a "dynamic" schema by setting all fields as a string.

getTotal

It receives filePath with the file path of the uploaded file

It returns an integer with the total number of lines in the file

Widget

By default the module accepts the following options:

• icon: please refer to the module options icons section. The icon is used as an icon when adding the widget to the page.
• className: CSS class name that will be passed to the widget template. Please refer to Setting a CSS class on core widgets
• placeholder: please refer to how to add placeholders to custom widgets
• placeholderClass: please refer to how to add placeholders to custom widgets
• widgetTemplates: an array of widget template names. Defaults to [ { value: 'table', label: 'apostropheDataSet:table' } ]. The editor will be given the opportunity to choose the template they want each time they add the widget. Templates are located in modules/@apostrophecms-pro/data-set-widget/views/. For table you'll have table.html. Please refer to the existing modules/@apostrophecms-pro/data-set-widget/views/table.html as reference. You can use this feature to define additional presentation choices beyond tables using your own markup.

The module does not provide any CSS for the table template in order to use the default HTML table CSS from your project.