Study guide · CLI, Scheduler, Message Bus and Mail
Background work: command line, scheduler, message bus, mail and upgrades
A lot of work in TYPO3 happens without anyone requesting a page. Our example is the extension my_extension again. It imports talks every night, creates a PDF in the background after each registration and sends it by email. In version 2 it ships an upgrade wizard.
The TYPO3 command line
The TYPO3 command line is built on Symfony Console. That is why commands follow the pattern area, colon, action, for example cache:flush. In a Composer installation you call the script typo3 in the folder vendor, subfolder bin.
vendor/bin/typo3 list
vendor/bin/typo3 help cache:flush
vendor/bin/typo3 extension:setup
vendor/bin/typo3 upgrade:run- The command
listshows all commands from the core and from extensions, grouped by prefix. - The command
helpfollowed by a command name shows its arguments and options. - The command
extension:setupcreates new tables and fields after you install or update extensions. It belongs in every deployment. - The command
upgrade:runexecutes upgrade wizards, andupgrade:listshows the open ones.
The command line needs no logged-in backend user and runs in every application context, including Production.
Your own command
Your own command extends the class Command from Symfony Console. You register it with the Symfony attribute AsCommand, which holds the name and a description. Alternatively you add it to Services.yaml with the tag console.command.
#[AsCommand(name: 'myextension:import-talks', description: 'Imports talks')]
final class ImportTalksCommand extends Command
{
public function __construct(private readonly TalkImporter $importer)
{
parent::__construct();
}
protected function configure(): void
{
$this->addArgument('conference', InputArgument::REQUIRED)
->addOption('dry-run', null, InputOption::VALUE_NONE);
}
protected function execute(InputInterface $input, OutputInterface $output): int
{
$io = new SymfonyStyle($input, $output);
$count = $this->importer->import((int)$input->getArgument('conference'));
$io->success($count . ' talks imported');
return Command::SUCCESS;
}
}Arguments and options are described in configure() with addArgument() and addOption(). Arguments are passed by position, and options get two dashes before their name. If a required argument is missing, the console stops before execute() runs. You read the values with getArgument() and getOption().
Output: execute() receives an InputInterface and an OutputInterface. Output goes through the output object, not through echo. SymfonyStyle is more convenient, with colored success messages, tables, progress bars and questions.
Verbosity: Symfony itself handles the switches for more output, that is one to three small v letters. You check them on the output with isVerbose(), isVeryVerbose() or isDebug(). They are not options of your command.
Return value: execute() returns an integer exit code. Command::SUCCESS is 0, Command::FAILURE is 1, and Command::INVALID is 2. Scripts and cron use it to see the result.
Services arrive through the constructor by dependency injection. Then you must call parent::__construct(). Otherwise Symfony stops with a message saying the command class is not correctly initialized.
Context: On the command line there is no request, no site, no TypoScript and no logged-in user. If you need page links, for example in a Fluid email, you have to create a request yourself.
The scheduler
The scheduler runs tasks on a timetable. Since TYPO3 14 you find it in the module group Administration, for administrators only. TYPO3 14 renamed the groups: the old group System is now Administration, and the old group Admin Tools is now System.
The scheduler does not start anything on its own. A cron job regularly calls the command scheduler:run, for example every five minutes. The scheduler then starts all due tasks. Page views or backend logins do not start any tasks.
When does a task run? It is single, at a fixed point in time, or recurring. A recurring task gets an interval in seconds, such as 3600 for hourly, or a cron expression, for example for daily at three in the morning. An end date can limit how long it keeps running.
Parallel runs are not allowed by default. If a task runs every five minutes but needs eight, the scheduler skips the next start.
Priority: Since TYPO3 14.2 every task has the priority high, normal or low. It decides the order before the time of the next execution does.
Commands as tasks, and your own tasks
Every command is schedulable. In the new task wizard of TYPO3 14, each console command appears as its own task type, grouped by the prefix before the colon. So cache:flush sits in the group cache. You fill in arguments and options in the task form. This turns our import into a nightly task without any scheduler code.
If a command should not be schedulable, you add the attribute AsNonSchedulableCommand to the class, since TYPO3 14. Before, you needed the entry schedulable with the value false in Services.yaml.
Your own task without a command is worth it when you need your own form fields:
- The class extends
AbstractTask. Its methodexecute()returns true on success. - In the TCA overrides of the table
tx_scheduler_task, you register it as its own type with the method addRecordType ofExtensionManagementUtility. - Your own fields are normal TCA columns. The task reads and writes them with
getTaskParameters()andsetTaskParameters().
Since TYPO3 14 every task is a TCA record with the fields tasktype and parameters, instead of a serialized PHP object. An upgrade wizard converts old tasks. The frequency options now come from TCA as well. Registering tasks through SC_OPTIONS with an AdditionalFieldProvider is deprecated.
Task or command? You can start a command directly on the command line and also in the scheduler. A classic task runs only in the scheduler and keeps its settings in the task record. When cron starts the scheduler, both run in the CLI context without a frontend request. When an administrator starts a task by hand in the module, however, it runs in that user's backend request. For new jobs a command is usually more flexible.
Synchronous or asynchronous
Synchronous means: the job is done right away, and the caller waits. Slow jobs therefore make the response slower.
Asynchronous means: the job goes into a queue as a message. A separate process, the worker, handles it later. The caller does not wait for the result.
Asynchronous processing suits long jobs whose result nobody needs right away: a large PDF export, sending mail, image processing or calls to external APIs. Login, rendering a page and checking a required field, on the other hand, need their result immediately.
Benefits: Slow or unreliable calls do not block the request. Load peaks and outages of external services are decoupled, so the system becomes more robust.
Challenges:
- A worker must run all the time or start regularly.
- Errors no longer show up in the original request and must be monitored separately.
- Messages can arrive more than once. Handlers should therefore be idempotent, that is, safe to run several times.
- The current request does not get the result, and the job is not processed any faster either.
The message bus
The TYPO3 message bus is built on Symfony Messenger.
The message is any simple PHP class without a base class. You send it through the Symfony MessageBusInterface, which you get by dependency injection. The sender does not know the handler. The handler carries the Symfony attribute AsMessageHandler and receives the message as a parameter.
final readonly class CreateCertificateMessage
{
public function __construct(public int $registrationUid) {}
}
// in the controller
$this->bus->dispatch(new CreateCertificateMessage($registration->getUid()));
#[AsMessageHandler]
final class CreateCertificateHandler
{
public function __invoke(CreateCertificateMessage $message): void
{
// load the registration, create the PDF, send the mail
}
}Alternatively you register the handler in Services.yaml. There you can also set the order of several handlers with before and after.
The default is synchronous. Without further configuration, TYPO3 routes all messages to the synchronous transport default. It handles them right away in the same request. If the handler is missing, TYPO3 throws an exception.
Asynchronous processing starts only with a route to an asynchronous transport. The transport doctrine stores messages in a table of the TYPO3 database. You write the route into the file additional.php in the folder config, subfolder system.
$GLOBALS['TYPO3_CONF_VARS']['SYS']['messenger']['routing'][CreateCertificateMessage::class] = 'doctrine';
// or for all messages:
$GLOBALS['TYPO3_CONF_VARS']['SYS']['messenger']['routing']['*'] = 'doctrine';The worker works through the queue. Without it, asynchronous messages just wait.
vendor/bin/typo3 messenger:consume doctrine --time-limit=3600Since TYPO3 14.2 the command has the Symfony limits for time, number of messages, failures and memory. With a limit, the worker ends regularly and frees its memory. A service manager such as systemd then restarts it.
Sending email
The Mail API builds messages with Symfony Mime and sends them with Symfony Mailer.
Settings apply to the whole installation. They live in the MAIL section of TYPO3_CONF_VARS, which you can edit in the module Settings in the group System. For different environments you set them in additional.php per application context.
- Transport: The default is sendmail. With smtp you send through a specific mail server, for example with
transport_smtp_server. The transport mbox writes all mails into the file named intransport_mbox_file, which is handy on development machines. You can also use your own class that implements the SymfonyTransportInterface, or a DSN. - Sender:
defaultMailFromAddressanddefaultMailFromNameapply when an extension sets no sender. - Spool: With the spool type memory, mails go out only at the end of the request. With file, TYPO3 collects them in a folder, and the command
mailer:spool:sendsends them regularly. This way no visitor waits for the mail server after submitting a form.
Testing: In the module Environment in the group System, “Test Mail Setup” sends a test mail through the configured transport.
Building and sending: In TYPO3 14, MailMessage is a pure data object. Like FluidEmail, it extends the Email class of Symfony Mime. You set the content with methods such as text() and html(). Sending always goes through the TYPO3 MailerInterface, which you get through the constructor. The old method send on the mail itself was removed in TYPO3 14.
$email = (new FluidEmail())
->from(new Address('team@example.org', 'Conference team'))
->to(new Address($attendee->getEmail(), $attendee->getName()))
->subject('Your registration')
->setTemplate('RegistrationConfirmation')
->assign('attendee', $attendee)
->format(FluidEmail::FORMAT_HTML)
->setRequest($request);
$email->attachFromPath($pdfPath, 'Certificate.pdf');
$this->mailer->send($email);- Addresses are described by the class
Address: first the address, then the display name. A plain text with just the address also works, then without a name. - Attachments:
attachFromPath()attaches a file, optionally with a name and a type.attach()takes content from memory. - FluidEmail looks for its template in the paths of the setting templateRootPaths in the MAIL section. You add your path there with a higher key, such as 700. Keys below 100 are reserved for the core.
- Format:
format()chooses HTML only, plain text only, or both. Both is the default. - Links in the template need the request from
setRequest(). Since TYPO3 14.2 theTemplatedEmailFactorycreates FluidEmails in one place.
Stepping in: Before sending, the mailer dispatches the BeforeMailerSentMessageEvent. A listener gets the message with getMessage() and changes it. This is how every mail on a test system gets a blind copy to the team, no matter which extension sends it. There is no setting for this. After sending comes the AfterMailerSentMessageEvent, for example for logging.
Upgrades and upgrade wizards
You try a major upgrade, for example from TYPO3 13 to 14, on a copy of the live system first. Before that, you back up the database and files.
Why deprecations first? TYPO3 marks API as deprecated in one version and removes it in the next major version. The deprecation log shows the affected calls. You do the same in your own extension, following Semantic Versioning. If a public method is to be replaced by a new one, you add the new one in a minor version. You mark the old one as deprecated and trigger a deprecation notice when it is called. You may remove it only in the next major version, because that is a breaking change.
Upgrade wizards adjust the data after an update. The database comparison only changes tables and columns, not their content. A wizard fits when existing records must be adapted after fields or values have changed. One example is switching old plugins from list_type to their own CType with the base class AbstractListTypeToCTypeUpdate. It does not fit work on every request or bulk editing by editors, because it runs once during the update.
Since TYPO3 14 the interface and the attribute live in the core, and the old locations in the extension install are deprecated.
#[UpgradeWizard('myExtension_moveVenueToLocation')]
final class MoveVenueToLocationWizard implements UpgradeWizardInterface
{
public function getTitle(): string { return 'Move venue into location'; }
public function getDescription(): string { return '…'; }
public function updateNecessary(): bool { /* are there old rows left? */ }
public function executeUpdate(): bool { /* rewrite */ return true; }
public function getPrerequisites(): array { return [DatabaseUpdatedPrerequisite::class]; }
}- The identifier comes from the attribute, not from a method.
updateNecessary()checks whether there is anything left to do. If it says no, the wizard is not offered.executeUpdate()migrates and returns true or false, not a count.getPrerequisites()lists the prerequisites. WithDatabaseUpdatedPrerequisite, the database schema is updated first.- With
RepeatableInterfacethe wizard is never marked as done and stays selectable.ConfirmableInterfaceasks first, andChattyInterfaceallows extra output.
On the command line you start wizards with upgrade:run. Undoing is not foreseen. Only the command upgrade:mark:undone resets the status.
Extension scanner, Rector and Fractor
The extension scanner sits in the module Upgrade in the group System, under “Scan Extension Files”. It searches the PHP code of extensions for places that the changelog lists as removed or deprecated, and links the entry. It works by static code analysis and runs nothing. False positives and missed places are therefore possible.
- Strong means: the match is certain, for example a known class name.
- Weak means: the code might be affected, for example because only the method name matches. You check such a match by hand. Calls through variables almost always give weak matches.
- The scanner does not recognize version checks in the code.
- A green result does not guarantee that the extension works.
Every changelog entry carries one of the tags FullyScanned, PartiallyScanned or NotScanned. They tell you whether the scanner finds the change. With NotScanned you check yourself.
// @extensionScannerIgnoreLine
$foo->someFalsePositiveMatch('foo');Rector rewrites PHP code based on rules, and TYPO3 Rector brings them for each TYPO3 version. Fractor does the same for TypoScript, FlexForms, Fluid and other files that are not PHP. PHPStan only analyzes, and PHP-CS-Fixer only formats. The command process with the option dry-run shows the changes of Rector only as a diff and writes nothing.
Run Rector and Fractor regularly. Then you remove deprecations early, the upgrade gets smaller, and each change stays easy to review. They do not replace tests and changelogs, though.
Notifications with actions
Notifications are short messages in the top right corner of the backend. You create them in JavaScript with the notification module of the backend, for example with the method success. They can get actions, which are buttons:
- The ImmediateAction runs a simple task right away and closes the notification.
- The DeferredAction is for longer tasks such as an AJAX request. Its callback returns a Promise, and until then the button shows a spinner. If someone closes the notification, the task still keeps running.
import Notification from '@typo3/backend/notification.js';
import ImmediateAction from '@typo3/backend/action-button/immediate-action.js';
Notification.success('Import finished', '42 talks imported', 0, [
{ label: 'Open log', action: new ImmediateAction(() => { /* open the module */ }) },
]);Key points
- Your own commands extend
Command, carryAsCommandand return an exit code. - The scheduler runs only when cron calls
scheduler:run. Parallel runs are off by default. - Every command can be scheduled as a task type, unless it carries
AsNonSchedulableCommand. - The message bus uses Symfony Messenger and is synchronous by default. Asynchronous handling needs a route and a worker.
- The
MailerInterfacesends mails, and the default transport is sendmail. - Upgrade wizards migrate data, the extension scanner finds deprecated PHP calls, and Rector and Fractor rewrite code.
Terms
- Exit code
- An integer with which a command reports success or failure.
- Handler
- A class that processes a message of the message bus.
- Idempotent
- Safe to run several times with the same result.
- Spool
- A buffer for mails that are sent later.
- Upgrade wizard
- A PHP class that adjusts data once after an update.
- Worker
- A process that works through a queue.