Study guide · Forms, records and CSV
Building forms, managing records, understanding CSV
On this page you follow Lena, an editor at the Weidenau City Library. She builds a contact form, manages records such as website users and events, and exchanges lists with other programs as CSV files. Tom is the administrator. Afterwards you know the form editor and the Records module with its clipboard and download, and you know why a “ü” sometimes turns into two strange characters.
Forms in TYPO3: the Forms module
The library needs forms for questions, for the reading circle and for a photo contest. In TYPO3 you build such forms with the Form Framework. It is a system extension of the Core and has been included since TYPO3 8. The administrator has to activate it and grant editors access to the module. Tom has done that, so Lena sees the module Forms in the Content area. It used to be called Web, Forms, and its overview of all forms is known as the Form Manager.
The Form Framework can do a lot:
- You build forms with the mouse, see a preview and reuse one form on many pages.
- It offers required fields, input checks, multi-step forms and automatic spam protection.
- After submission, actions called finishers run, for example an email to the editors.
It also has limits. Formatting of field labels, multilingual support and the textual design of emails are limited. Since TYPO3 14.2, the email finishers can include a custom formatted message, and some fields get a rich text editor. For heavily designed emails or many languages, you still need help from a developer.
Creating a new form
In the Forms module, Lena clicks Create new form. A wizard guides her through a few steps:
- She chooses how to start: a blank form, in the wizard “Blank form”, or a predefined form, “Predefined form”. TYPO3 ships a simple contact form as a template, the “Simple contact form”.
- She chooses where the form is stored. Since TYPO3 14.2 the database is available, and it is the recommended storage. Saving it as a YAML file in a file storage is deprecated. If only one storage is allowed, the wizard skips this step.
- She enters a name she will recognize later, such as “Library contact”. It only appears in the backend. For a predefined form, she also picks the template here.
- She checks her settings and creates the form. The form editor then opens.
In the editor, Lena adds fields with Create new element. With drag and drop she reorders existing fields. On the right, the inspector holds the settings of the selected field. At the top of the tree view sits the form name. Clicking it opens the general form settings, including the finishers. The preview icon shows a simple preview.
In the preview she notices a heading “Step”. A form consists of at least one step, and this title is rendered too. For a single-step form it makes no sense. The heading should come from the page anyway. So Lena clicks “Step” in the tree on the left and deletes the word.
Form elements and their settings
There is an element for every kind of input. The most important ones:
- Text is a single-line field, for example for the name.
- Textarea is a multi-line field for longer input such as a message.
- Email address, Telephone number and URL are text fields for exactly that kind of data.
- Number and Date take numbers and dates.
- Checkbox is a single box. Multi checkbox, Radio button, Single select and Multi select offer choices.
- File upload and Image upload accept files.
- Hidden is a field visitors do not see. Static text only displays text.
There are also containers. A Fieldset groups related fields under a heading, for example street, postcode and town of a postal address. Screen readers announce this group as a unit, which improves accessibility. Placing fields side by side in columns is the job of the Grid: Row element instead, a grid row with columns.
Most elements share the same basic settings:
- Label is the visible caption of the field.
- Description is a help text below the field.
- Field visible shows or hides the field.
- Placeholder shows an example in the empty field.
- Default value is a value that is already in the field.
- The checkbox Field required makes the field mandatory. The editors' documentation also calls this Mandatory field or Required field. It comes with a Custom error message.
- Validators check the input, more on that in a moment.
- Autocomplete tells the browser what kind of data is requested, for example a first name or an email address.
A required field must be filled in before the form can be sent. If the input is missing, the error message appears and the form is not submitted.
Placeholder or default value? The placeholder is only a grey hint, such as a sample phone number. It disappears as soon as you type and is never submitted. The default value, in contrast, is a real, prefilled value. It stays in the field until you change it, and otherwise it is submitted as it is.
Validators: checking input
A validator checks whether an input is valid. It does not translate or store anything. If the input is invalid, the form shows an error message, which you can customize as well. The form editor offers these validators, among others:
- Alphanumeric allows only letters and digits. Non-XML text forbids HTML code.
- String length limits the length of a text.
- Email checks for a valid email address.
- Integer number, Floating-point number and Number check numbers, Number range checks a range of numbers.
- Regular expression checks against a pattern you define.
- Date range checks a range of dates.
- Number of submitted values counts the chosen options of a multiple choice.
- File size and Number of files check uploaded files.
The Email address element brings its validator along: Email is active automatically, and it is the only one you can choose there. On smartphones, a keyboard with the at sign appears.
Uploads. In the photo contest “My favourite book”, participants may upload up to three pictures. Since TYPO3 14.2, upload fields accept several files. Lena ticks Allow multiple upload on the Image upload element and adds the validator Number of files with a maximum of three. The File size validator limits the size of each file.
Uploads have a privacy side. In the field Uploads save path you choose the folder where the files end up. If that folder is publicly reachable, anyone can open the files, and search engines may index them. The files are not encrypted or deleted automatically. For sensitive documents such as job applications, Lena asks Tom to set up a protected storage path.
Finishers: what happens after submission
A finisher is an action that runs after someone has submitted the form. It is not a field, not a button and not a validator. Lena picks it in the general form settings. The choices are:
- Email to receiver (you) sends an email to the website operator, at the library to the information desk mailbox.
- Email to sender (form submitter) sends an email to the person who filled in the form, for example a receipt.
- Confirmation message shows a thank-you text on the page after submission.
- Redirect to a page sends the visitor to another page.
- Delete uploads deletes uploaded files, for example after they were sent by email.
For the email finishers, Lena enters subject, recipient, name and a copy address. A button with a plus sign inserts values from the form, for example first and last name as the sender's name.
The order matters. Finishers run exactly in the order in which they appear in the list. TYPO3 does not reorder them. The redirect ends the execution, and no finisher runs after it. That is why Redirect to a page always goes last.
Adding the form to a page
A finished form does not appear anywhere yet. Lena adds it like this:
- She opens the Layout module and selects the page “Contact” in the page tree.
- She creates a new content element and chooses the element Form in the tab “Form elements”.
- In the Plugin tab she selects the form definition “Library contact”.
- She saves the content element and checks the page in the frontend.
Because the form definition is stored separately from the content element, Lena can reuse the same form on many pages. On “Contact”, “Media” and “Opening hours” she adds one Form element each, all with the same definition. She enters the heading in the regular header field of each content element. That is why she cleared the “Step” title in the form. She does not need copies of the form definition. Changes to the definition apply everywhere it is used. In the plugin you can also override some finisher settings for that one page, for example the recipient.
Records in the Records module
Not everything in TYPO3 is a page or a content element. Website users, user groups, categories or news are records. Many of them have no module of their own. Then you work in the module Records in the Content area. Up to TYPO3 13 it was called Web, List.
Lena selects a page or a folder in the page tree, for example the folder “Website users”. Records shows all records stored there, grouped by type. At the top of the module there are buttons, for example to create a new record and to show the search and the clipboard.
- Open and edit: A click on the title or on the pencil icon opens the record's form.
- Hide and delete: Next to it are icons to hide, unhide and delete. More actions are in the menu with the three dots.
- Search: In long lists, the search box helps. You set how many levels of subpages it includes.
Languages. Since TYPO3 14, Records has a language menu at the top right. Lena ticks the languages the list should show. If she chooses French, she sees the French translations below their originals. Records of the default language always stay in the list, and so do records with the language “All languages”. TYPO3 remembers the choice per page, and the Layout module shows the same languages.
Copying and moving. You copy a single record through the context menu. Lena opens it with the three dots or the icon to the left of the record and chooses Copy, or Cut to move it. At the target she inserts it with Paste. TYPO3 hides copied content elements at first. If an element with the same title already exists at the target, TYPO3 appends “(copy 1)” to the title of the copy.
Two messages when pasting are worth knowing:
- Wrong place: Some records may only be stored in folders, for example website users and website user groups. If Lena pastes a user group on a standard page, an error message appears and nothing is pasted. She switches to the right folder and pastes it there.
- Unique values: If Lena copies the website user “max” into the same folder, the copy is called “max1”. User names must be unique within the same folder, so TYPO3 appends a number. Lena opens the copy and enters a suitable name.
Clipboard and mass editing
For several records at once, there is the clipboard. Lena shows it through the View menu at the top of the Records module. The clipboard now appears at the bottom. The normal clipboard holds only one record. So Lena chooses a numbered clipboard such as Clipboard 1. Only with a numbered clipboard can she put several records on the clipboard at once.
There is a checkbox in front of each record. As soon as records are checked, the list shows new buttons for the selection:
- Edit edits all selected records.
- Delete deletes them.
- Transfer to clipboard puts them on the clipboard. This button only exists with a numbered clipboard.
- Remove from clipboard takes them off again, also only with a numbered clipboard.
The clipboard collects records, even from different pages. By default, the clipboard has Move elements selected. If you want to copy, click Copy elements. Then the original page stays unchanged. On the target page, Lena clicks “Paste in clipboard content” and confirms the dialog.
Columns and mass editing. The list shows only a few fields per table. With Show columns, Lena opens a dialog, picks more fields and clicks Update. The fields now appear as columns. To select all records she chooses Check all in the selection menu on the left. Toggle selection inverts the selection. Then she has two ways:
- Edit shows all fields of all selected records, one after the other.
- Edit columns shows only the fields that are currently displayed as columns.
Showing a single table. In the overview of all tables, Records shows 20 entries per table by default. A click on List only this table at the title of a table shows only that table, with 100 entries per page. List all tables takes you back. An integrator can change both values with TSconfig.
Downloading records: CSV and JSON
Records offers a download for every table. The dialog knows exactly two formats: CSV and JSON. Lena sets:
- Columns: only the selected columns or all columns.
- Format of values: Raw values are the raw database values, such as numeric IDs or timestamps. Processed values are prepared for humans, with readable labels and dates as in the backend.
- Filename: free to choose.
- Format and the options that depend on it.
For CSV, Lena chooses the delimiter: comma, semicolon or pipe. The default is the comma. She also chooses the quote character: double quote, single quote or space. The default is the double quote. An integrator can set a character of their own with TSconfig. It then appears in the dialog as an extra choice, “user defined”, and is preselected. For JSON, the option Meta information is offered instead. It decides whether all additional information, only the table name and page ID, or nothing at all comes before the records.
Understanding CSV files
CSV stands for Character Separated Values. A CSV file is a plain text file with table data. Each line is one record, and a delimiter separates the values. Often the first line holds the column names. CSV is used to exchange data between systems, for example between TYPO3 and a spreadsheet.
Common delimiters are comma and semicolon, as well as tab and the vertical bar, the pipe. You can open and edit CSV with any text editor, or neatly in columns with a spreadsheet.
The enclosure. What if a value contains the delimiter itself? Then an enclosure, usually a double quote, surrounds the whole value. The delimiter inside no longer counts. If a quote appears inside the value, it is doubled, and the whole value goes in quotes.
1;"Muster; Partner GmbH";Berlin
"She said ""Hello"""Character encoding: why Grün sometimes turns into nonsense
Computers store only numbers. A character encoding maps characters to numbers. It defines which number stands for “A”, for “ä” or for the euro sign.
Why are there several encodings? Early encodings covered only a few languages. ASCII knows 128 characters, without umlauts. ISO-8859-1, also called Latin-1, uses one byte per character and knows 256 characters. That is enough for Western European languages, but not for Chinese or Greek. Windows-1252 is a close relative of Latin-1. Unicode collects almost all scripts of the world. The encoding UTF-8 can represent all Unicode characters and uses one to four bytes per character. The first 128 characters are identical to ASCII. UTF-8 is the standard on the web today, and TYPO3 works with UTF-8 too.
What goes wrong. If a file is read with a different encoding than the one it was written in, umlauts and special characters appear garbled. The data is not lost, it is just read with the wrong table.
So if an imported list shows a capital A with tilde in the middle of “Grün”, the text was saved in UTF-8 and read as ISO-8859-1. Lena opens the file again and chooses UTF-8.
Finding the encoding. Many text editors show the encoding of a file in the status bar and can change it. On the command line, the program file helps. Renaming the file extension changes nothing.
Converting the encoding. The library network sends Lena a media list as CSV in Windows-1252. The website and TYPO3 work with UTF-8. So Lena converts the file to UTF-8 before processing it, with an editor or a tool such as the import dialog of her spreadsheet. Using it unchanged would produce broken umlauts.
Key points
- You build forms in the Forms module. Since TYPO3 14.2 they are best stored in the database.
- Every field needs a clear label. Validators check the input.
- Finishers run in their order. The redirect ends everything and goes last.
- You embed one form definition on many pages with the Form content element.
- Records without their own module are managed in Records. The clipboard moves by default.
- Edit shows all fields, Edit columns only the displayed columns.
- The download offers CSV and JSON. UTF-8 is the standard for characters.
Terms
- Form Framework
- The system extension form, which editors use to build forms in the Forms module.
- Form definition
- The stored form, which you embed on any number of pages with the Form content element.
- Finisher
- An action after submission, such as an email, a confirmation or a redirect.
- Validator
- A check whether the input in a field is valid.
- Records
- The module in the Content area that shows all records of a page or folder grouped by type.
- CSV
- A text file with table data whose values are separated by a delimiter on each line.
- Enclosure
- Usually a quote character that surrounds a value containing a delimiter.
- Character encoding
- The mapping of characters to numbers, today mostly UTF-8.