Study guide · Extbase: Persistence and Validation
Storing data and validating input with Extbase
A visitor registers for a conference. How does their data get into the table, and how does Extbase make sure it is correct? This page explains both: storing and loading objects, and validating input. Controllers, views and plugins are covered in the course „Extbase: MVC, Plugins und Views“.
How Extbase stores data
Extbase contains an ORM, an object-relational mapper. It translates between PHP objects and table rows. You work with objects and repositories instead of SQL. The repository is your interface. The persistence manager collects changes and writes them in one go. The identity map remembers loaded objects: the same record gives you the same object within one request.
Our example is the extension my_extension with the vendor MyVendor. It has a model Conference with a title, a start date, a city and speakers. Speakers and attendees share one person table.
The model and its table
A model lives in Classes/Domain/Model and extends AbstractEntity. Each instance is one record. The base class provides uid and pid, together with getUid(), getPid() and setPid(). Before the first save, getUid() returns null. You declare every other field yourself, including system fields such as crdate or sorting.
class Conference extends AbstractEntity
{
#[Validate(validator: 'NotEmpty')]
protected string $title = '';
protected ?\DateTime $startDate = null;
protected string $city = '';
}Properties are protected, not private. Extbase fills them through a method of the parent class. That method cannot write private properties of the child class. They silently keep their default value.
Names by convention: For the table name, the vendor is dropped. The remaining parts of the class name are lowercased and joined with underscores. So Conference becomes the table tx_myextension_domain_model_conference. A class name such as PostComment is not split, it becomes postcomment. Properties are different: Extbase converts camelCase to snake_case, so conferenceDate maps to the column conference_date. In Fluid you keep using the property name.
Nothing works without TCA. Extbase reads columns and relations from the TCA of the table. If a column has no TCA definition, the property stays empty, even though the column exists. TYPO3 derives the database columns from the TCA. You only need ext_tables.sql for special cases such as custom indexes.
Deviations go into Classes.php in Configuration/Extbase/Persistence. There you map a model to a foreign table such as fe_users, or a property to a column with another name. Models that follow the conventions do not need this file. Since TYPO3 14, the Configuration module of the lowlevel extension shows the merged mapping of all extensions.
Types: If a column stores a point in time as an integer, for example with the TCA type datetime, use a nullable DateTime. Extbase converts it for you.
Entity or value object? An entity has its own identity, its UID. Conference 42 stays the same conference, even with a new title. A value object is defined only by its values, such as an amount of money. You do not change it, you replace it. Since TYPO3 14 you write value objects as plain PHP classes, because AbstractValueObject is now internal.
Relations and lazy loading
For a relation to one object, use a typed property, for example a nullable Location. For several objects, use the Extbase ObjectStorage. The similar PHP class SplObjectStorage does not work: Extbase cannot load or save relations with it.
The ObjectStorage has attach() to add, detach() to remove and contains() to check. It has no methods called add or remove.
/** @var ObjectStorage<Speaker> */
protected ObjectStorage $speakers;
public function __construct()
{
$this->initializeObject();
}
public function initializeObject(): void
{
$this->speakers = new ObjectStorage();
}
public function addSpeaker(Speaker $speaker): void
{
$this->speakers->attach($speaker);
}
public function removeSpeaker(Speaker $speaker): void
{
$this->speakers->detach($speaker);
}Lazy loading: The attribute Lazy loads a relation only on first access. For an ObjectStorage, Extbase puts a LazyObjectStorage in as a placeholder. Saving works as usual. For a relation to a single object, the type must also allow LazyLoadingProxy.
#[Lazy]
protected Location|LazyLoadingProxy|null $location = null;Two more attributes work on properties. Cascade only supports the value remove: it deletes dependent objects when you remove the parent through Extbase. It has no effect when an editor deletes in the backend, because the DataHandler does not know it. Transient excludes a property completely. Extbase never reads or writes it, which suits calculated values.
Saving with the persistence manager
The persistence manager writes new, changed and removed objects to the database with persistAll(). The Extbase bootstrap calls it automatically after every run of a plugin or backend module. That also applies when the action returns a redirect.
Extbase detects changes itself. If you change a loaded object with a setter, persistAll() writes exactly the changed columns. You do not need update(). This is called dirty tracking.
Deleting: If the table has a delete field under ctrl.delete, remove() only sets the field deleted to one, just like the backend. Without that field, the row really disappears.
You save manually when you need the new UID straight away. The same goes for console commands or scheduler tasks, where no Extbase bootstrap runs. For this, inject the PersistenceManagerInterface.
The interface API also includes isNewObject(), getIdentifierByObject() and clearState(). To ask whether an object is new, use isNewObject(), not the internal method on the object. clearState() forgets all loaded objects and saves memory during large imports.
New in TYPO3 14: The reference index is always updated on save, and the option updateReferenceIndex is gone. The identity map is language aware: loaded with different language settings, an object is a separate instance. The feature toggle extbase.enableHistoryTracking writes changes to sys_history. It is off by default. For a single table, you switch it off again in the TCA with ctrl.extbase.enableHistoryTracking set to false.
The standard repository
Conference comes with the class ConferenceRepository in Classes/Domain/Repository. It extends Repository and is often empty. It is the only way to the database for this model.
Built in for reading are findAll(), findByUid(), findByIdentifier(), findBy(), findOneBy(), count() and countAll(). For writing there are add(), update(), remove() and removeAll(), but no save or delete.
The magic methods are gone. Methods such as findByTitle or countByOnline were deprecated since TYPO3 12.3 and are removed in TYPO3 14. Today you pass the criteria as an array. findBy() also takes an ordering, a limit and an offset. count() takes the same criteria, while countAll() counts everything.
$this->conferenceRepository->findBy(
['city' => 'Berlin'],
['startDate' => QueryInterface::ORDER_ASCENDING],
10,
);
$this->conferenceRepository->findOneBy(['slug' => $slug]);
$this->conferenceRepository->count(['online' => true]);Ordering: Without any setting there is no ORDER BY, and the database decides the order. Extbase does not apply default_sortby from the TCA. You set a default ordering with the property defaultOrderings in the repository, or at runtime with setDefaultOrderings(). The keys are property names, so startDate. The direction comes from the constants ORDER_ASCENDING and ORDER_DESCENDING of the QueryInterface.
Getting the repository: The controller receives it through constructor injection, as a constructor parameter with protected readonly. If the constructor is already taken, use a method such as injectConferenceRepository(). The ObjectManager has been gone since TYPO3 12. Using GeneralUtility::makeInstance() is wrong too. It loses settings of the shared instance, such as query settings. And the dependencies of the repository are not wired correctly.
Custom queries
Custom queries are repository methods whose names start with find or count. Each one starts with createQuery(). You set a constraint with matching() and run the query with execute().
public function findUpcomingInCity(string $city): QueryResultInterface
{
$query = $this->createQuery();
$query->matching(
$query->logicalAnd(
$query->equals('city', $city),
$query->greaterThanOrEqual('startDate', new \DateTime('today')),
)
);
$query->setOrderings(['startDate' => QueryInterface::ORDER_ASCENDING]);
$query->setLimit(10);
return $query->execute();
}Exactly one constraint: matching() expects exactly one constraint. You combine several with logicalAnd(), logicalOr() or logicalNot(). If you pass two arguments, only the first one counts. A second call to matching() replaces the first, and the method does not accept an array. For „online or in Berlin“ you need logicalOr().
equals()compares with exactly one value, without wildcards.like()only searches for part of a text when a percent sign stands before and after the search term.in()checks whether the value is in a list, for example a UID in a list of UIDs.contains()checks whether a relation contains a specific object.
All arguments are property names, not columns. You follow relations with a dot: speakers.name finds conferences with a speaker of that name. Extbase builds the joins itself.
Limit and result: setLimit() limits the number, setOffset() sets the starting point. A limit of zero throws an exception. execute() returns a QueryResult with objects, and with the argument true it returns raw rows. getFirst() returns the first object or null, and toArray() returns mapped objects. count() on the query counts without loading objects.
Since TYPO3 14, orderBy() and addOrderBy() can also sort with the expressions concat(), trim() and coalesce(). To paginate, create a QueryResultPaginator from the result, the current page and the items per page. A SimplePagination provides the page numbers for the template. The old Fluid widgets have been gone since TYPO3 11.
Extbase or Doctrine? Extbase queries use properties, return objects and add storage pages, visibility and language on their own. The Doctrine QueryBuilder uses columns and returns arrays. For sums, grouping or mass updates, use the QueryBuilder. More on this in the course „Datenbankzugriff und DataHandler“.
Storage pages, visibility and language
Without further configuration, every repository query in the frontend is restricted automatically. It only searches the storage pages. It respects the enable fields hidden, start time, end time and user group. It excludes deleted records and respects the current language. It does not filter by owner.
You set the storagePid in TypoScript under plugin.tx_myextension with persistence.storagePid. The field Record Storage Page in the plugin overrides this value. Subfolders are included with persistence.recursive. Only findByUid() and findByIdentifier() ignore the storagePid. New records go to the first storage page. For another folder, set newRecordStoragePid for the model class under persistence.classes.
Query settings come from getQuerySettings() on the query:
setRespectStoragePage()with false switches off the storage pages.setIgnoreEnableFields()with true ignores the enable fields. WithsetEnableFieldsToBeIgnored()and the name disabled, only hidden is skipped. Without that list, start and end time are skipped too.setIncludeDeleted()andsetLanguageAspect()control deleted records and language.
One method or all of them? Settings on a single query apply only to that query. That is how only findFeatured() returns conferences from all folders. For all methods, create the settings in a public method initializeObject() of the repository and set them with setDefaultQuerySettings(). Or override createQuery().
Since TYPO3 14.3, Extbase follows the fallbackType of the site language. With strict, the default, records without a translation are missing.
Single table inheritance
With single table inheritance, several classes of one inheritance tree share a table. A type column says which class a row is. In our example, Speaker and Attendee extend the class Person, and all of them live in one person table.
- In the TCA, you define the type column under
ctrl.type. Extbase reads it from there. - In
Classes.php, you list the subclasses under subclasses on the parent model. - Each subclass gets the same tableName and its own recordType in
Classes.php.
return [
Person::class => [
'subclasses' => [Speaker::class, Attendee::class],
],
Speaker::class => [
'tableName' => 'tx_myextension_domain_model_person',
'recordType' => 'speaker',
],
// Attendee::class the same way with 'recordType' => 'attendee'
];The old configuration through TypoScript no longer exists. The repository of Person restricts the query to its own recordType and those of its subclasses. It does not return other types. When loading, Extbase checks the type column of each row: a row of type speaker becomes a Speaker object. A repository of a subclass filters only on that type.
Validation in Extbase
Before an action runs, Extbase checks its arguments.
Validation runs for every action with typed arguments, not only for forms. Anything that is not a validator on the model or the parameter is not checked by anyone.
Attributes only: Since TYPO3 14, Extbase silently ignores old annotations in doc comments. Then no validation happens at all. The attributes live in the Extbase namespace Attribute. The old namespace Annotation is deprecated and goes away with TYPO3 15. You name the validator as a string or a class name and pass options as named arguments. A single array argument and the argument param are deprecated.
#[Validate(validator: 'NotEmpty')]
#[Validate(validator: 'StringLength', options: ['minimum' => 3, 'maximum' => 50])]
protected string $title = '';
public function createAction(
#[Validate(validator: ConferenceDateValidator::class)]
Conference $conference,
): ResponseInterfaceBuilt-in validators are named by short names such as NotEmpty, StringLength or EmailAddress. There is no attribute called Required. Almost all validators let empty values pass. So an empty field with only EmailAddress is valid. For „required and well formed“, combine NotEmpty with the format validator.
Symfony constraints also work since TYPO3 14, for example the constraint Iban on a property. Extbase already ships the package symfony/validator. Messages starting with LLL are translated, and Extbase converts the Symfony placeholders. The constraints File and Image do not work yet.
Showing errors: The ViewHelper f:form.validationResults provides the errors of a field. Its attribute for names the path, for example conference.title.
Injected fields: The Fluid form writes a hidden field __trustedProperties. It lists all fields and is signed with an HMAC. On submit, Extbase only maps these fields. So nobody can inject a field such as featured.
Forms, errors and access
When validation fails, Extbase calls errorAction(). It adds an error message and forwards, within the same request, to the action that showed the form. Input and errors are available there, and the status is 400. The message „An error occurred while trying to call“ comes from getErrorFlashMessage(). If that method returns false, the message is dropped.
The new and create pattern: The newAction shows the form, the createAction saves. If create fails, the errorAction leads back to new. So that new accepts the invalid object with its input, its parameter has the attribute IgnoreValidation. Without it, new would fail too. Since TYPO3 14 it sits directly on the parameter. The variant with argumentName above the method is deprecated.
public function newAction(#[IgnoreValidation] ?Conference $conference = null): ResponseInterface
public function createAction(Conference $conference): ResponseInterfaceAccess: Since TYPO3 14, the attribute Authorize on an action method checks the login, groups or a callback method. With requireGroups, one of the groups is enough. Several Authorize attributes on one action must all be met. A callback method in the controller must be public. Whether a record belongs to the user is something you check yourself with a callback. A refusal returns status 403 through the ErrorController.
Rate limit: RateLimit limits requests per IP address, by default five in 15 minutes. Above that, Extbase answers with status 429. It only works reliably for uncached actions, because otherwise the page cache answers before the controller runs.
Custom validators and validation in TYPO3
A custom validator lives in Classes/Validation/Validator, for example as SeatCountValidator. It usually extends AbstractValidator and needs no registration. You only implement isValid(). The method returns nothing. You report errors with addError(), with a message and a unique code, usually a timestamp. For a single field there is addErrorForProperty(). Extbase ignores return values and exceptions. Do not declare the class final, so others can extend it.
class SeatCountValidator extends AbstractValidator
{
protected $supportedOptions = [
'minimum' => [10, 'Minimum number of seats', 'integer'],
];
protected function isValid(mixed $value): void
{
if ($value < $this->options['minimum']) {
$this->addError('Not enough seats', 1729500000);
}
}
}So options are declared in supportedOptions, and the resolved values are in options. You attach the validator with its full class name. A short name such as SeatCount only looks among the core validators and throws a NoSuchValidatorException. If a validator implements ValidatorInterface directly, TYPO3 14 also requires setRequest() and getRequest().
Where does it go? A validator on a property only sees that value. Rules across several fields belong on the action parameter. Example: if a conference starts in less than four weeks, a speaker is required. The validator needs the date and the speaker, so the whole object.
Validation across TYPO3 happens in three independent places:
- In the TCA, with required or eval, for backend forms and the DataHandler. You make a backend field required with required set to true on the column. NOT NULL in the database does not prevent an empty text field.
- In Extbase, through validators on models and action parameters.
- In the Form Framework, through the validators of the form elements.
Checks in the browser, such as the HTML attribute required, are only a convenience. They are easy to bypass, and requests can be sent without any form. That is why the server always checks too.
Key points
- Writing only happens at
persistAll(), which Extbase calls after every plugin run. - Extbase saves changed objects without
update(). A new UID only exists after saving. - Properties are protected and need a TCA column.
- Deviating names and single table inheritance go into
Classes.php. - Instead of magic finders, use
findBy(),findOneBy()andcount()with an array. - An empty list is usually a storagePid problem.
matching()takes exactly one constraint.- Only PHP attributes with named arguments count.
- Custom validators extend
AbstractValidatorand are attached by their full class name. - TCA and Extbase validate separately, and the server always checks.
Terms
- Dirty tracking
- Extbase detects changed properties itself and writes only those.
- Enable fields
- Visibility fields such as hidden, start time, end time and user group.
- Identity map
- Register of loaded objects, so the same record gives the same object.
- ObjectStorage
- Collection of objects for relations to several objects.
- ORM
- Object-relational mapping, the translation between objects and table rows.
- Property mapping
- Converting request data into PHP types and objects.
- Single table inheritance
- Several classes in one table, told apart by a type column.
- storagePid
- Page or pages where a repository looks for records.