Lecture 5 – Data sources

• working together on a project • common needs, common resource • data as a product • data-based service

• new combinations of data • new relationships of data • new visualisations of data • new understanding of data

data that is freely available to everyone

Its abundance and complexity (many datasets, with different definitions and access points)

data which is in a format that can be understood by a computer (JSON, XML, csv)

system for annotating a document in a away that is synthetically distinguishable from the text (e.g. HTML, XML)

file format whose specification describes how different elements of data coexist in a computer file

structured data that describes other data can be … descriptive (title, author, file size), structural (relationships, chapters, elements of JSON), administrative (version number, archiving data, createDate)

provides a standard language for describing a predictive model that can be passed between analytic software

path of current directory

change directory to DIRPATH

output the filenames of DIRPATH

copy FILENAME to NEWFILENAME

rename FILENAME to NEWFILENAME

output TEXT

output content of FILENAME

output the content of FILENAME, but one screen at a time Advantage: faster access because if file is large it doesn’t access the complete file, but accesses it page by page.

count number of characters, term, lines in FILENAME Lines , word, number of bytes for text file

output lines in FILENAME that match PATTERN

output the first lines of FILENAME

output the last lines of FILENAME

process text files in various ways

• access • format • value & vocabulary • metadata • software • process & workflow

• establishing the standards • enacting the standards