Skip to content

Commit 999492b

Browse files
committed
Keep libgit2 loaded with php-fpm through LibGit2DriverFactory::preload()
PHP unloads a library at the end of every request, so each php-fpm request loaded and initialized libgit2 again: about 10 ms, mostly OpenSSL reading the CA certificates. Called from opcache.preload, preload() loads libgit2 into an FFI scope that lives as long as the process, with a header generated for the library path (FFI::load() reads it from the header), and initializes it before php-fpm forks its workers: a request then gets it in 0.1 ms. It also declares the classes of the driver while preloading, which may then call FFI with the default ffi.enable=preload. Checked with forked workers of the built-in server: TLS (an HTTPS remote) works after the fork.
1 parent 0d960d5 commit 999492b

5 files changed

Lines changed: 161 additions & 29 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ Changes since [v1.6.0](https://github.com/gitonomy/gitlib/releases/tag/v1.6.0).
1414
`Repository`/`Admin`. It avoids spawning a `git` process for every operation; see
1515
["Choosing a driver"](doc/repository.md#choosing-a-driver) for its requirements, its
1616
differences with the `'cli'` driver, and when it is faster. `benchmark.php` compares both
17-
drivers on a repository. ([#253](https://github.com/gitonomy/gitlib/pull/253))
17+
drivers on a repository. With php-fpm, `LibGit2DriverFactory::preload()` keeps libgit2 loaded
18+
from `opcache.preload`. ([#253](https://github.com/gitonomy/gitlib/pull/253))
1819
- The `'driver'` option also accepts a configured driver factory (`Driver\Cli\CliDriverFactory`,
1920
`Driver\Ffi\LibGit2DriverFactory`, or any `Driver\DriverFactoryInterface`), to configure the
2021
backend or to replace it in tests; see

‎doc/repository.md‎

Lines changed: 23 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -140,12 +140,25 @@ Differences with the `'cli'` driver:
140140
- Rename detection in diffs may pair files differently.
141141
- Blame line ranges are limited to `<start>,<end>` and `<start>,+<count>`.
142142

143-
The `ffi.enable` PHP setting defaults to `preload`: outside the CLI
144-
(php-fpm...), only preloaded code can use FFI, and the `'ffi'` driver throws
145-
an `UnsupportedOperationException`. Either preload gitlib's sources through
146-
[`opcache.preload`](https://www.php.net/manual/en/opcache.preloading.php), or
147-
set `ffi.enable` to `true`, which lets any code of the application call C
148-
functions.
143+
#### With php-fpm
144+
145+
Call `LibGit2DriverFactory::preload()` from the
146+
[`opcache.preload`](https://www.php.net/manual/en/opcache.preloading.php)
147+
script, with the library path you give to `LibGit2DriverFactory`, if any:
148+
149+
```php
150+
// preload.php
151+
require __DIR__.'/vendor/autoload.php';
152+
153+
Gitonomy\Git\Driver\Ffi\LibGit2DriverFactory::preload();
154+
```
155+
156+
libgit2 then stays loaded and initialized for the lifetime of php-fpm,
157+
instead of every request loading it again (about 10 ms, see below), and the
158+
driver works with the default `ffi.enable` setting, `preload`: outside the
159+
CLI, only preloaded code can use FFI. Without preloading, the `'ffi'` driver
160+
throws an `UnsupportedOperationException` unless `ffi.enable` is `true`,
161+
which lets any code of the application call C functions.
149162

150163
### Performance
151164

@@ -180,9 +193,10 @@ an AMD Ryzen AI 9 HX 370.
180193
- The `'ffi'` driver reads objects (commits, files, references) 2 to 28
181194
times faster: no process to start.
182195
- Loading libgit2 costs about 10 ms, mostly libgit2 setting up OpenSSL,
183-
which reads the system's CA certificates. PHP frees it at the end of
184-
every request: with php-fpm, each request pays it again, so a request
185-
running a few operations isn't faster. A worker or a command pays it once.
196+
which reads the system's CA certificates. A command or a worker pays it
197+
once. PHP unloads libgit2 at the end of every request: with php-fpm, each
198+
request pays it again, unless
199+
[libgit2 is preloaded](#with-php-fpm), which brings it down to 0.1 ms.
186200
- libgit2 doesn't use git's commit-graph, and caches objects per
187201
repository only: walking a long history (counting commits, finding the
188202
branches including a commit, logging a path) is up to 60 times slower.

‎src/Gitonomy/Git/Driver/Ffi/LibGit2.php‎

Lines changed: 86 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ final class LibGit2
4545

4646
/**
4747
* Loads libgit2 once per request (the whole process with the CLI) and per
48-
* library path.
48+
* library path, or takes the one preload() loaded.
4949
*
5050
* @throws UnsupportedOperationException The FFI extension or libgit2 1.9 is missing, or "ffi.enable" restricts FFI
5151
*/
@@ -56,6 +56,40 @@ public static function load(?string $libraryPath): \FFI
5656
return $ffiByLibraryPath[$libraryPath ?? ''] ??= self::doLoad($libraryPath);
5757
}
5858

59+
/**
60+
* Loads and initializes libgit2 for the whole process, from the
61+
* opcache.preload script: PHP unloads a library at the end of every
62+
* request otherwise, and libgit2 initializes itself again (about 10 ms).
63+
*
64+
* @throws UnsupportedOperationException The FFI extension or libgit2 1.9 is missing
65+
*/
66+
public static function preload(?string $libraryPath): void
67+
{
68+
self::assertFfiIsAvailable();
69+
$path = $libraryPath ?? self::locate();
70+
self::assertSupportedVersion($path);
71+
72+
// FFI::load() reads the library to load from the header.
73+
$header = tempnam(sys_get_temp_dir(), 'gitlib-libgit2-');
74+
75+
try {
76+
file_put_contents($header, \sprintf("#define FFI_SCOPE \"%s\"\n#define FFI_LIB \"%s\"\n", self::scope($libraryPath), $path).file_get_contents(__DIR__.'/libgit2.h'));
77+
$ffi = \FFI::load($header);
78+
} catch (\FFI\Exception $e) {
79+
throw new UnsupportedOperationException(\sprintf('Unable to preload libgit2 from "%s".', $path), 0, $e);
80+
} finally {
81+
unlink($header);
82+
}
83+
84+
// FFI::load() only warns while preloading.
85+
if (null === $ffi) {
86+
throw new UnsupportedOperationException(\sprintf('Unable to preload libgit2 from "%s".', $path));
87+
}
88+
89+
// Once for the forked processes (php-fpm workers...) too.
90+
self::initialize($ffi);
91+
}
92+
5993
/**
6094
* @throws DriverException When $rc is a libgit2 error code
6195
*/
@@ -87,6 +121,45 @@ public static function oidToHex(\FFI $ffi, CData $oid): string
87121
}
88122

89123
private static function doLoad(?string $libraryPath): \FFI
124+
{
125+
self::assertFfiIsAvailable();
126+
127+
$ffi = self::preloaded($libraryPath) ?? self::define($libraryPath ?? self::locate());
128+
// Only counts one more user once libgit2 is initialized.
129+
self::initialize($ffi);
130+
131+
return $ffi;
132+
}
133+
134+
/**
135+
* @return \FFI|null What preload() loaded for $libraryPath, if it was called
136+
*/
137+
private static function preloaded(?string $libraryPath): ?\FFI
138+
{
139+
try {
140+
return \FFI::scope(self::scope($libraryPath));
141+
} catch (\FFI\Exception) {
142+
return null;
143+
}
144+
}
145+
146+
private static function define(string $libraryPath): \FFI
147+
{
148+
self::assertSupportedVersion($libraryPath);
149+
150+
try {
151+
return \FFI::cdef(file_get_contents(__DIR__.'/libgit2.h'), $libraryPath);
152+
} catch (\FFI\Exception $e) {
153+
throw new UnsupportedOperationException(\sprintf('Unable to load libgit2 from "%s"; use the "cli" driver.', $libraryPath), 0, $e);
154+
}
155+
}
156+
157+
private static function scope(?string $libraryPath): string
158+
{
159+
return 'gitonomy/gitlib:'.($libraryPath ?? '');
160+
}
161+
162+
private static function assertFfiIsAvailable(): void
90163
{
91164
if (!\extension_loaded('ffi')) {
92165
throw new UnsupportedOperationException('The "ffi" driver requires the FFI extension; use the "cli" driver.');
@@ -96,36 +169,32 @@ private static function doLoad(?string $libraryPath): \FFI
96169
\FFI::cdef();
97170
} catch (\FFI\Exception $e) {
98171
// ffi.enable defaults to "preload": outside the CLI, only preloaded code can use FFI.
99-
throw new UnsupportedOperationException('The "ffi.enable" setting restricts the FFI API: set it to "true", preload gitlib, or use the "cli" driver.', 0, $e);
172+
throw new UnsupportedOperationException('The "ffi.enable" setting restricts the FFI API: call LibGit2DriverFactory::preload() from the opcache.preload script, set "ffi.enable" to "true", or use the "cli" driver.', 0, $e);
100173
}
174+
}
101175

102-
$libraryPath ??= self::locate();
103-
176+
private static function assertSupportedVersion(string $libraryPath): void
177+
{
104178
try {
105-
$versionFfi = \FFI::cdef('int git_libgit2_version(int *major, int *minor, int *rev);', $libraryPath);
179+
$ffi = \FFI::cdef('int git_libgit2_version(int *major, int *minor, int *rev);', $libraryPath);
106180
} catch (\FFI\Exception $e) {
107181
throw new UnsupportedOperationException(\sprintf('Unable to load libgit2 from "%s"; use the "cli" driver.', $libraryPath), 0, $e);
108182
}
109183

110-
$major = $versionFfi->new('int');
111-
$minor = $versionFfi->new('int');
112-
$rev = $versionFfi->new('int');
113-
$versionFfi->git_libgit2_version(\FFI::addr($major), \FFI::addr($minor), \FFI::addr($rev));
184+
$major = $ffi->new('int');
185+
$minor = $ffi->new('int');
186+
$rev = $ffi->new('int');
187+
$ffi->git_libgit2_version(\FFI::addr($major), \FFI::addr($minor), \FFI::addr($rev));
114188
if (self::SUPPORTED_VERSION !== [$major->cdata, $minor->cdata]) {
115189
throw new UnsupportedOperationException(\sprintf('The "ffi" driver requires libgit2 %s, but "%s" is libgit2 %d.%d.%d; use the "cli" driver.', implode('.', self::SUPPORTED_VERSION), $libraryPath, $major->cdata, $minor->cdata, $rev->cdata));
116190
}
191+
}
117192

118-
try {
119-
$ffi = \FFI::cdef(file_get_contents(__DIR__.'/libgit2.h'), $libraryPath);
120-
} catch (\FFI\Exception $e) {
121-
throw new UnsupportedOperationException(\sprintf('Unable to load libgit2 from "%s"; use the "cli" driver.', $libraryPath), 0, $e);
122-
}
123-
193+
private static function initialize(\FFI $ffi): void
194+
{
124195
if ($ffi->git_libgit2_init() < 0) {
125196
throw new UnsupportedOperationException(\sprintf('Unable to initialize libgit2: %s', self::lastErrorMessage($ffi)));
126197
}
127-
128-
return $ffi;
129198
}
130199

131200
private static function locate(): string

‎src/Gitonomy/Git/Driver/Ffi/LibGit2DriverFactory.php‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,26 @@ public function __construct(?string $libraryPath = null)
4343
$this->ffi = LibGit2::load($libraryPath);
4444
}
4545

46+
/**
47+
* To call from the opcache.preload script, with php-fpm for instance:
48+
* libgit2 then stays loaded and initialized for the whole process,
49+
* instead of every request loading it again (about 10 ms), and the driver
50+
* may call FFI whatever "ffi.enable" is.
51+
*
52+
* @param string|null $libraryPath The libgit2 shared library, as given to the constructor
53+
*
54+
* @throws UnsupportedOperationException The FFI extension or libgit2 1.9 is missing
55+
*/
56+
public static function preload(?string $libraryPath = null): void
57+
{
58+
LibGit2::preload($libraryPath);
59+
60+
// Classes declared while preloading may call FFI, even when
61+
// "ffi.enable" is "preload", its default.
62+
class_exists(LibGit2Driver::class);
63+
class_exists(PathFilter::class);
64+
}
65+
4666
public static function isSupported(): bool
4767
{
4868
try {

‎tests/Gitonomy/Git/Tests/LibGit2DriverFactoryTest.php‎

Lines changed: 30 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212

1313
namespace Gitonomy\Git\Tests;
1414

15+
use Gitonomy\Git\Driver\Ffi\LibGit2DriverFactory;
1516
use PHPUnit\Framework\Attributes\RequiresPhpExtension;
1617
use PHPUnit\Framework\TestCase;
1718
use Symfony\Component\Process\Process;
@@ -21,6 +22,8 @@
2122
*/
2223
class LibGit2DriverFactoryTest extends TestCase
2324
{
25+
private const string AUTOLOAD = __DIR__.'/../../../../vendor/autoload.php';
26+
2427
#[RequiresPhpExtension('ffi')]
2528
public function testRestrictedFfiApiIsReported(): void
2629
{
@@ -34,9 +37,34 @@ public function testRestrictedFfiApiIsReported(): void
3437
}
3538
PHP;
3639

37-
$process = new Process([\PHP_BINARY, '-d', 'ffi.enable=0', '-r', $code, __DIR__.'/../../../../vendor/autoload.php']);
40+
$process = new Process([\PHP_BINARY, '-d', 'ffi.enable=0', '-r', $code, self::AUTOLOAD]);
41+
$process->mustRun();
42+
43+
$this->assertSame('The "ffi.enable" setting restricts the FFI API: call LibGit2DriverFactory::preload() from the opcache.preload script, set "ffi.enable" to "true", or use the "cli" driver.', $process->getOutput());
44+
}
45+
46+
#[RequiresPhpExtension('ffi')]
47+
#[RequiresPhpExtension('Zend OPcache')]
48+
public function testPreloadKeepsLibgit2LoadedForTheProcess(): void
49+
{
50+
if (!LibGit2DriverFactory::isSupported()) {
51+
$this->markTestSkipped('libgit2 1.9 is not installed.');
52+
}
53+
54+
$preload = AbstractTestCase::createTempDir().'/preload.php';
55+
file_put_contents($preload, \sprintf('<?php require %s; %s::preload();', var_export(self::AUTOLOAD, true), LibGit2DriverFactory::class));
56+
57+
// FFI::scope() throws unless preload() registered the scope of libgit2.
58+
$code = <<<'PHP'
59+
require $argv[1];
60+
FFI::scope('gitonomy/gitlib:');
61+
echo new Gitonomy\Git\Repository($argv[2], ['driver' => 'ffi'])->getHeadCommit()->getHash();
62+
PHP;
63+
64+
$repository = AbstractTestCase::createFoobarRepository();
65+
$process = new Process([\PHP_BINARY, '-d', 'opcache.enable_cli=1', '-d', 'opcache.preload='.$preload, '-r', $code, self::AUTOLOAD, $repository->getGitDir()]);
3866
$process->mustRun();
3967

40-
$this->assertSame('The "ffi.enable" setting restricts the FFI API: set it to "true", preload gitlib, or use the "cli" driver.', $process->getOutput());
68+
$this->assertSame($repository->getHeadCommit()->getHash(), $process->getOutput());
4169
}
4270
}

0 commit comments

Comments
 (0)