diff --git a/CHANGELOG.md b/CHANGELOG.md index 5e9ba811..46ec41e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -185,6 +185,24 @@ because it turns other people's test suites red. BeautifulSoup on html5lib count the rules as text of the page. No rule names anything to fetch. Files made with the earlier settings keep their bytes. +- **HTML: links, and an indented page.** `links` puts a list of that many + links above the pictures and the blocks. They go through 18 kinds in turn - + a link to the list itself, relative, to the folder above, from the root, a + query alone, absolute with a query and a fragment, `http`, without a + scheme, percent encoded, an international domain, a port, `mailto:`, + `tel:`, `ftp:`, a download, a new window, `nofollow` and `data:` - and the + text of each says which it is. Every address is on a domain kept for + examples. The Nu HTML Checker finds nothing wrong with any of them, and + jsdom, lxml, BeautifulSoup and Python's `urljoin` resolve them alike. + DOMPurify with its defaults takes out the `data:` address and the target + that opens a new window, and it ran out of 8 GB of memory over a million + links, which lxml read in 3 seconds. The manifest records `links`. + `formatting=indented` puts two spaces in front of a line for every element + it sits in and leaves the text of `pre` as it is - Chromium places every + element of it where it places them on the page of lines, and the Nu HTML + Checker finds nothing wrong with it. With `nesting` the spaces grow with + every level, so `nesting` 1000 makes a page of at least 1.9 MB. Files made + with the earlier settings keep their bytes. - **A length of time as a setting.** `duration`, `change_interval` and `keyframe_interval` take `10s`, `1m30s`, `1h`, `500ms` or `59.9s`, to the millisecond. A bare number is refused rather than read as seconds, and so is a length that does not @@ -299,6 +317,9 @@ because it turns other people's test suites red. ### Fixed +- **The window called the byte order mark setting "Bom".** It reads "BOM" now, + on `txt`, `md`, `xml` and `html` alike. + - **Ctrl+C waited for a file that holds thousands of files to be planned.** A zip, a tar.gz, a mail message or a page with ten thousand small pictures inside took over a minute to plan, and a run stopped in that minute went diff --git a/README.md b/README.md index 7b31e827..9bec5ad7 100644 --- a/README.md +++ b/README.md @@ -634,7 +634,7 @@ recipe. `tfg formats ` prints the allowed range or list for each: | `geojson` | `geometry`, `formatting`, `precision`, `altitude`, `vertices`, `holes`, `winding`, `antimeridian`, `unlocated`, `ids`, `bbox` | | `eml` | `body`, `text_encoding`, `charset`, `headers`, `recipients`, `line_endings`, `attachments`, `attachment_format`, `attachment_size`, `filename_style`, `message_depth`, `inline`, `inline_format`, `inline_size` | | `svg` | `width`, `height` | -| `html` | `structure`, `nesting`, `characters`, `elements`, `bom`, `formatting`, `styles`, `inline`, `inline_format`, `inline_size` | +| `html` | `structure`, `nesting`, `characters`, `elements`, `links`, `bom`, `formatting`, `styles`, `inline`, `inline_format`, `inline_size` | | `yaml`, `toml` | none in this build - the document is a fixed shape, and a size is the only thing to ask for | ``` diff --git a/internal/format/htmlfile/blocks.go b/internal/format/htmlfile/blocks.go index 799e3cde..f12cf178 100644 --- a/internal/format/htmlfile/blocks.go +++ b/internal/format/htmlfile/blocks.go @@ -22,21 +22,23 @@ import ( // or a list cut in the middle still renders, and it says something other than // it meant. type blocks struct { - // tail is what the closing record ends with, which is the only thing the - // shape of the file changes down here. - tail string // m is how the blocks are spelled. The words drawn are the same in every // spelling, so one seed fills a page of one line and a page of many with // the same blocks. m *markup + // lead is the indentation of a block and inner the one of a list's items + // and a table's rows, both empty unless the page is indented. + lead, inner string } -// Shortest is the smallest closing record: an empty paragraph plus whatever -// this shape closes after it. -func (b blocks) Shortest() int64 { return int64(len(b.m[paraOpen]) + len(b.tail)) } +// Shortest is the smallest closing record: an empty paragraph. +func (b blocks) Shortest() int64 { + return int64(len(b.lead) + len(b.m[paraOpen]) + len(b.m[paraClose])) +} func (b blocks) Append(dst []byte, rng *rand.Rand) []byte { m := b.m + dst = append(dst, b.lead...) switch rng.IntN(5) { case 0: dst = append(dst, m[h2Open]...) @@ -45,14 +47,19 @@ func (b blocks) Append(dst []byte, rng *rand.Rand) []byte { case 1: dst = append(dst, m[ulOpen]...) for i, n := 0, 2+rng.IntN(3); i < n; i++ { + dst = append(dst, b.inner...) dst = append(dst, m[liOpen]...) dst = appendPhrase(dst, rng, 3+rng.IntN(4)) dst = append(dst, m[liClose]...) } + dst = append(dst, b.lead...) return append(dst, m[ulClose]...) case 2: dst = append(dst, m[tableOpen]...) + dst = append(dst, b.inner...) + dst = append(dst, m[tableHead]...) for i, n := 0, 2+rng.IntN(3); i < n; i++ { + dst = append(dst, b.inner...) dst = append(dst, m[rowOpen]...) dst = strconv.AppendInt(dst, int64(i+1), 10) dst = append(dst, m[cellSep]...) @@ -61,6 +68,7 @@ func (b blocks) Append(dst []byte, rng *rand.Rand) []byte { dst = append(dst, words[rng.IntN(len(words))]...) dst = append(dst, m[rowClose]...) } + dst = append(dst, b.lead...) return append(dst, m[tableClose]...) case 3: // A real page carries entities, and an ampersand left raw is the @@ -84,10 +92,11 @@ func (blocks) Discard() {} func (b blocks) AppendExact(dst []byte, rng *rand.Rand, n int64) []byte { start := len(dst) + dst = append(dst, b.lead...) dst = append(dst, b.m[paraOpen]...) - used := int64(len(dst)-start) + int64(len(b.tail)) + used := int64(len(dst)-start) + int64(len(b.m[paraClose])) dst = appendFiller(dst, n-used) - return append(dst, b.tail...) + return append(dst, b.m[paraClose]...) } func appendPhrase(dst []byte, rng *rand.Rand, n int) []byte { diff --git a/internal/format/htmlfile/catalogue.go b/internal/format/htmlfile/catalogue.go index b8c24f7e..acc37e0e 100644 --- a/internal/format/htmlfile/catalogue.go +++ b/internal/format/htmlfile/catalogue.go @@ -150,10 +150,28 @@ func (s settings) catalogueHeadFor() string { return s.spelling()[headCatalogue] } -// catalogueBlocks is what the catalogue adds to the body. -func (s settings) catalogueBlocks() string { - if !s.every { - return "" +// appendCatalogue appends what the catalogue adds to the body, at the depth of +// its content. +func (c content) appendCatalogue(dst []byte, lead string) []byte { + s := c.s + switch { + case !s.every: + return dst + case s.indented(): + return appendLaidOut(dst, s.laid().catalogue, lead) + } + return append(dst, s.spelling()[bodyCatalogue]...) +} + +// catalogueBytes is what appendCatalogue appends, worked out without building +// it. +func (c content) catalogueBytes() int64 { + s := c.s + switch { + case !s.every: + return 0 + case s.indented(): + return laidOutBytes(s.laid().catalogue, s.leadBytes()) } - return s.spelling()[bodyCatalogue] + return int64(len(s.spelling()[bodyCatalogue])) } diff --git a/internal/format/htmlfile/characters.go b/internal/format/htmlfile/characters.go index 67ce624d..baa1f0bd 100644 --- a/internal/format/htmlfile/characters.go +++ b/internal/format/htmlfile/characters.go @@ -29,11 +29,28 @@ const ( `and entities & < > "   © € € 😀.

` + "\n" ) -// internationalBlocks is what the setting adds, nothing for ascii, in the -// spelling of the page. -func (s settings) internationalBlocks() string { - if !s.international { - return "" +// appendInternational appends what the setting adds, nothing for ascii, in the +// spelling of the page and at the depth of its content. +func (c content) appendInternational(dst []byte, lead string) []byte { + s := c.s + switch { + case !s.international: + return dst + case s.indented(): + return appendLaidOut(dst, s.laid().scripts, lead) } - return s.spelling()[scripts] + return append(dst, s.spelling()[scripts]...) +} + +// internationalBytes is what appendInternational appends, worked out without +// building it. +func (c content) internationalBytes() int64 { + s := c.s + switch { + case !s.international: + return 0 + case s.indented(): + return laidOutBytes(s.laid().scripts, s.leadBytes()) + } + return int64(len(s.spelling()[scripts])) } diff --git a/internal/format/htmlfile/content.go b/internal/format/htmlfile/content.go new file mode 100644 index 00000000..3c3b5988 --- /dev/null +++ b/internal/format/htmlfile/content.go @@ -0,0 +1,156 @@ +package htmlfile + +import ( + "context" + "io" + + "github.com/donislawdev/TestingFilesGenerator/internal/core" +) + +// content is what a page writes after its label - the levels of nesting, the +// paragraphs in other scripts, the catalogue, the links, the blocks and what +// closes them - each one written from the settings and counted without being +// built. Split from settings on 2026-10-14, when indentation and links took +// that type past the methods one type may carry (typeshape_test.go): settings +// says what was asked for, and this writes it. +type content struct{ s settings } + +// content is the writing of a page of these settings. +func (s settings) content() content { return content{s} } + +// chunks builds what a page repeats and writes it each time it passes +// writeChunk, asking between two writes whether the run was stopped. After the +// first error it builds nothing more, and close says what the error was. +type chunks struct { + ctx context.Context + w io.Writer + buf []byte + err error +} + +func newChunks(ctx context.Context, w io.Writer) *chunks { + return &chunks{ctx: ctx, w: w, buf: make([]byte, 0, writeChunk+1024)} +} + +// add appends two pieces, one after the other - most often an indentation and +// what stands at it. +func (c *chunks) add(first, second string) { + if c.err != nil { + return + } + c.buf = append(append(c.buf, first...), second...) + if len(c.buf) >= writeChunk { + c.err = flush(c.ctx, c.w, c.buf) + c.buf = c.buf[:0] + } +} + +// close writes what is left, or says why nothing more was written. +func (c *chunks) close() error { + if c.err != nil { + return c.err + } + return core.WriteAll(c.w, c.buf) +} + +// flush writes what was built so far, after asking whether the run was +// interrupted. +func flush(ctx context.Context, w io.Writer, buf []byte) error { + if err := ctx.Err(); err != nil { + return err + } + return core.WriteAll(w, buf) +} + +// levelIndent is the indentation of the level k levels in, cut from the +// deepest one rather than built for every level, nothing unless the page is +// indented. +func (c content) levelIndent(in indents, k int) string { + return in.lead[:min(len(in.lead), int(indentationBytes(c.s.bodyDepth()+k)))] +} + +// levelsBytes is what the levels take opened, or closed - each one at the depth +// it opens at, which an indented page writes in front of it. Indented, a level +// k deep has 2(b+k) spaces, so the sum is worked out rather than added up. +func (c content) levelsBytes(piece string) int64 { + s := c.s + n := int64(s.nesting) + total := n * int64(len(piece)) + if s.indented() { + b := int64(s.bodyDepth()) + total += int64(len(indentUnit)) * (n*b + n*(n-1)/2) + } + return total +} + +// writeOpening writes what follows the label: the levels of nesting, then the +// paragraphs in other scripts and the catalogue. The levels are written a +// chunk at a time - a hundred thousand of them indented are 10 GB. +func (c content) writeOpening(ctx context.Context, w io.Writer, in indents) error { + s, m := c.s, c.s.spelling() + out := newChunks(ctx, w) + for k := 0; k < s.nesting && out.err == nil; k++ { + out.add(c.levelIndent(in, k), m[divOpen]) + } + if out.err == nil { + out.buf = c.appendInternational(out.buf, in.lead) + out.buf = c.appendCatalogue(out.buf, in.lead) + } + return out.close() +} + +// openingBytes is what writeOpening writes, worked out without building it. +func (c content) openingBytes() int64 { + return c.levelsBytes(c.s.spelling()[divOpen]) + c.internationalBytes() + c.catalogueBytes() +} + +// writeClosing writes what closes the levels from the innermost out, then the +// body and the document. It follows the last block rather than ending it, so a +// page closing a hundred thousand indented levels never builds them as one +// record. +func (c content) writeClosing(ctx context.Context, w io.Writer, in indents) error { + s, m := c.s, c.s.spelling() + out := newChunks(ctx, w) + for k := s.nesting - 1; k >= 0 && out.err == nil; k-- { + out.add(c.levelIndent(in, k), m[divClose]) + } + out.add("", c.documentClose()) + return out.close() +} + +// closingBytes is what writeClosing writes, worked out without building it. +func (c content) closingBytes() int64 { + return c.levelsBytes(c.s.spelling()[divClose]) + int64(len(c.documentClose())) +} + +// documentClose closes the body and the document of a page, and nothing in a +// fragment, which has neither. +func (c content) documentClose() string { + s := c.s + if s.shape != structureDocument { + return "" + } + closing := s.spelling()[bodyClose] + if s.indented() { + return joined(layOut(closing, s.bodyDepth())) + } + return closing +} + +// blocks is the body builder for these settings. +// +// What closes the levels, the body and the document follows the last record +// (writeClosing) and is counted out of the bytes the records fill, so a change +// to it has to reach both. Until 2026-10-14 it sat in the last record, and the +// golden hashes - a fragment and a page nested on one line among them - hold +// the bytes to what they were then. +func (c content) blocks(in indents) blocks { + return blocks{m: c.s.spelling(), lead: in.lead, inner: in.inner} +} + +// shortestRecord is the smallest closing record, worked out without building +// the indentation: an empty paragraph at the depth of the content. +func (c content) shortestRecord() int64 { + m := c.s.spelling() + return c.s.leadBytes() + int64(len(m[paraOpen])+len(m[paraClose])) +} diff --git a/internal/format/htmlfile/html.go b/internal/format/htmlfile/html.go index b3a0d4cb..6ffd7aa9 100644 --- a/internal/format/htmlfile/html.go +++ b/internal/format/htmlfile/html.go @@ -56,9 +56,9 @@ func init() { // than a comment. Label: format.LabelVisible, Oracle: "python-html", - // Links come later, if at all (docs/HTML-2-2026-10-14.md section 3, - // B4). Declaring only what is here is what makes a recipe asking for - // them fail loudly instead of quietly producing something else. + // Declaring only what is here is what makes a recipe asking for + // something else fail loudly instead of quietly producing a page + // without it. Properties: properties(), GeneratorVersion: generatorVersion, Generator: generator{}, @@ -88,7 +88,7 @@ func (generator) PlanContext(ctx context.Context, r format.Request) (format.Plan return format.Plan{}, err } - pictures, pictureBytes, err := planPictures(ctx, r, s.pictures, s.spelling()) + pictures, pictureBytes, err := planPictures(ctx, r, s) if err != nil { return format.Plan{}, err } @@ -155,9 +155,10 @@ func belowMinimum(s settings, requested, min int64) error { // the reason names, whatever else was asked. reason = core.Says("html.MinimumWithPictures", "the file holds whole blocks and every picture in base64, which is about a third larger than the picture itself, and together they need that much") case s.adds(): - reason = core.Says("html.MinimumWithSettings", "the file holds whole blocks and what %s, %s, %s, %s and %s add to them, and together they need that much", + reason = core.Says("html.MinimumWithSettings", "the file holds whole blocks and what %s, %s, %s, %s, %s, %s and %s add to them, and together they need that much", core.A("Nesting", core.LabelTerm(settingNesting)), core.A("Characters", core.LabelTerm(settingCharacters)), - core.A("Elements", core.LabelTerm(settingElements)), core.A("BOM", core.LabelTerm(textenc.SettingBOM)), + core.A("Elements", core.LabelTerm(settingElements)), core.A("Links", core.LabelTerm(settingLinks)), + core.A("BOM", core.LabelTerm(textenc.SettingBOM)), core.A("Formatting", core.LabelTerm(settingFormatting)), core.A("Styles", core.LabelTerm(settingStyles))) case s.shape == structureFragment: reason = core.Says("html.MinimumFragment", "a fragment holds whole blocks, and one of them needs that much") @@ -182,6 +183,10 @@ func labelledHead(s settings, r format.Request, min int64) (head, heading string label := core.Label("html", r.Bytes, r.Seed) m := s.spelling() heading = m[labelOpen] + label + m[labelClose] + if s.indented() { + // The label sits in the body, outside the levels. + heading = indentation(s.bodyDepth()) + heading + } plain := s.prologue() head, extra := plain, int64(0) @@ -211,16 +216,27 @@ func (generator) Write(ctx context.Context, w io.Writer, p format.Plan) error { // The label sits outside the levels of nesting, so a reader that drops // elements past its depth - libxml2 does, measured on 2026-10-14 - still // shows which file it read. - head := m.head + m.labelLine + m.settings.opening() + c, in := m.settings.content(), m.settings.indents() + head := m.head + m.labelLine if err := core.WriteAll(w, []byte(head)); err != nil { return err } - if err := writePictures(ctx, w, m.pictures); err != nil { + if err := c.writeOpening(ctx, w, in); err != nil { + return err + } + if err := c.writeLinks(ctx, w, in); err != nil { + return err + } + if err := writePictures(ctx, w, m.pictures, in.lead); err != nil { return err } rng := core.NewRand(m.seed) - return core.FillRecords(ctx, w, rng, p.Bytes-int64(len(head))-m.pictureBytes, m.settings.blocks()) + filled := p.Bytes - int64(len(head)) - c.openingBytes() - c.linksBytes() - m.pictureBytes - c.closingBytes() + if err := core.FillRecords(ctx, w, rng, filled, c.blocks(in)); err != nil { + return err + } + return c.writeClosing(ctx, w, in) } // minimumBytes is the smallest whole page, which is what the registry declares. @@ -228,9 +244,11 @@ func (generator) Write(ctx context.Context, w io.Writer, p format.Plan) error { // do. func minimumBytes() int64 { return minimumFor(settings{shape: structureDocument}) } -// minimumFor is the prologue, what the settings open the body with and one -// whole block, computed from the pieces Write uses rather than written down so -// it cannot drift away from the template. +// minimumFor is the prologue, what the settings open the body with, one whole +// block and what closes the levels and the document, computed from the pieces +// Write uses rather than written down so it cannot drift away from the +// template. func minimumFor(s settings) int64 { - return int64(len(s.prologue())) + s.openingBytes() + int64(len(s.spelling()[paraOpen])) + s.tailBytes() + c := s.content() + return int64(len(s.prologue())) + c.openingBytes() + c.linksBytes() + c.shortestRecord() + c.closingBytes() } diff --git a/internal/format/htmlfile/indent.go b/internal/format/htmlfile/indent.go new file mode 100644 index 00000000..6ed4a00a --- /dev/null +++ b/internal/format/htmlfile/indent.go @@ -0,0 +1,151 @@ +package htmlfile + +import ( + "regexp" + "strings" +) + +// An indented page: every line of a page of lines, indented two spaces for +// each element open where it starts - the width JSON indents with, and the +// children of html indented as Prettier lays them out. Nothing else about the +// lines changes, so an indented page is a page of lines with spaces in front +// of them (docs/HTML-5-2026-10-14.md section 2.2). Measured on 2026-10-14, +// Chromium places every element of the two the same. +// +// The pieces written once - the head, the paragraphs in other scripts and the +// catalogue - are indented here by the elements they open. The pieces a page +// repeats - the levels of nesting, the blocks, the links and the pictures - +// sit at a depth only the settings know, and take their indentation from it +// as they are written, so a hundred thousand levels are never built in memory. + +// indentUnit is one level of indentation. +const indentUnit = " " + +// indentation is the indentation of a line depth levels deep. +func indentation(depth int) string { return strings.Repeat(indentUnit, depth) } + +// indentationBytes is its length, worked out without building it. +func indentationBytes(depth int) int64 { return int64(depth * len(indentUnit)) } + +// line is one line of a piece laid out by its elements, without its line +// break. A line inside pre is text rather than layout, and takes no +// indentation at all - not its own and not the depth it is written at. +type line struct { + text string + pre bool +} + +// voidElements have no end tag, so they open nothing. +var voidElements = map[string]bool{ + "area": true, "base": true, "br": true, "col": true, "embed": true, "hr": true, "img": true, + "input": true, "link": true, "meta": true, "source": true, "track": true, "wbr": true, +} + +// tag finds a start or an end tag: the slash, the name, and what follows the +// name up to the bracket. The doctype starts with "!" and is not one. +var tag = regexp.MustCompile(`<(/?)([a-zA-Z][a-zA-Z0-9-]*)([^>]*)>`) + +// layOut indents every line of text by the depth of the elements open where it +// starts, beginning at depth: a line that starts by closing one stands at the +// depth of its start tag. text ends with a line break, the way every piece of +// a page of lines does. +func layOut(text string, depth int) []line { + var out []line + if text == "" { + // Nothing is no lines, not one empty one - a fragment with no sheet + // has no prologue, and would otherwise open with a blank line. + return out + } + inPre := false + for _, raw := range strings.Split(strings.TrimSuffix(text, "\n"), "\n") { + out = append(out, lineAt(raw, depth, inPre)) + depth, inPre = after(raw, depth, inPre) + } + return out +} + +// lineAt is one line written depth levels deep, or as it is inside pre. +func lineAt(raw string, depth int, inPre bool) line { + if inPre { + return line{text: raw, pre: true} + } + if strings.HasPrefix(raw, "link to this list`, + `relative link`, + `link to the folder above`, + `link from the root`, + `link with only a query`, + `absolute link with a query and a fragment`, + `http link`, + `link without a scheme`, + `percent encoded link`, + `international domain link`, + `link with a port`, + `mail link`, + `phone link`, + `ftp link`, + `download link`, + `link to a new window`, + `nofollow link`, + `data link`, +} + +// linksDetail is what both surfaces say about the setting. +const linksDetail = "How many links the page carries, in a list above the pictures and the blocks. " + + "They go through 18 kinds in turn and then repeat - a link to the list itself, relative, to the folder above, from the root, a query alone, " + + "absolute with a query and a fragment, http, without a scheme, percent encoded, an international domain, a port, " + + "mailto:, tel:, ftp:, a download, a new window, nofollow and data: - and the text of each says which it is. " + + "Every address is on a domain kept for examples. DOMPurify with its defaults takes out the data: address and the target that opens a new window, " + + "and it ran out of 8 GB of memory over a million links, which lxml read in 3 seconds. " + + "The catalogue of every element has six links of its own, which this does not count." + +// linksBytes is what the list takes, worked out without building it: the list +// around its items, and the items, the kinds in turn. +func (c content) linksBytes() int64 { + s := c.s + if s.links == 0 { + return 0 + } + m, lead := s.spelling(), s.leadBytes() + n := int64(s.links) + var cycle, part int64 + for i, k := range linkKinds { + cycle += int64(len(k)) + if i < s.links%len(linkKinds) { + part += int64(len(k)) + } + } + item := s.innerBytes() + int64(len(m[liOpen])+len(m[liClose])) + return 2*lead + int64(len(m[linksOpen])+len(m[linksClose])) + n*item + n/int64(len(linkKinds))*cycle + part +} + +// writeLinks writes the list a chunk at a time, so a million links never sit in +// memory and an interrupted run stops between two chunks. +func (c content) writeLinks(ctx context.Context, w io.Writer, in indents) error { + s, m := c.s, c.s.spelling() + if s.links == 0 { + return nil + } + out := newChunks(ctx, w) + out.add(in.lead, m[linksOpen]) + for i := 0; i < s.links && out.err == nil; i++ { + out.add(in.inner, m[liOpen]) + out.add(linkKinds[i%len(linkKinds)], m[liClose]) + } + out.add(in.lead, m[linksClose]) + return out.close() +} diff --git a/internal/format/htmlfile/markup.go b/internal/format/htmlfile/markup.go index d3baf7bc..edbfe166 100644 --- a/internal/format/htmlfile/markup.go +++ b/internal/format/htmlfile/markup.go @@ -46,13 +46,20 @@ const ( liOpen liClose ulClose + // The table opens on a line of its own and its head row on the next, so + // an indented page can put the row a level further in. tableOpen + tableHead rowOpen cellSep rowClose tableClose quoteOpen quoteClose + // The list of links around its items, which are spelled as the items of + // any list. + linksOpen + linksClose // The paragraphs in other scripts, and a picture's line around its name // and its base64. scripts @@ -95,14 +102,20 @@ var plain = markup{ liOpen: "
  • ", liClose: "
    • \n", ulClose: "\n", - tableOpen: "\n\n", + tableOpen: "
    idfieldvalue
    \n", + tableHead: "\n", rowOpen: "\n", // The table closes and the quote holds a paragraph. - tableClose: "
    idfieldvalue
    ", cellSep: "", rowClose: "
    \n", - quoteOpen: "

    ", - quoteClose: "

    \n", + tableClose: "\n", + quoteOpen: "

    ", + quoteClose: "

    \n", + // The id is what the first link points at, and one no document property + // has: DOMPurify 3.4.17 takes out an id that would shadow one - links and + // forms both, measured on 2026-10-14 - and the link would point nowhere. + linksOpen: `\n", scripts: international, pictureLead: `

    `, pictureTail: `

    ` + "\n", @@ -249,16 +262,17 @@ func (s settings) styleName() string { return s.styles } -// spelling is how these settings spell a page. +// spelling is how these settings spell a page. An indented page is spelled as +// a page of lines and indented as it is written. func (s settings) spelling() *markup { - return spellings[spellingKey{s.minified, s.styleName()}] + return spellings[spellingKey{s.minified(), s.styleName()}] } // lineEnding is what the manifest says the lines of the page end with: none for // a page on one line, which has no line to end, where lf would describe a file // that is not there. func (s settings) lineEnding() string { - if s.minified { + if s.minified() { return "none" } return "lf" diff --git a/internal/format/htmlfile/page.go b/internal/format/htmlfile/page.go index b0ebf906..e80dc9a2 100644 --- a/internal/format/htmlfile/page.go +++ b/internal/format/htmlfile/page.go @@ -1,12 +1,10 @@ package htmlfile -import "strings" - // How a page is laid out from its settings, split out of html.go on 2026-10-14. // The order is the one measured and decided in docs/HTML-2-2026-10-14.md // section 12: the mark, the skeleton and the head, the label, the levels of -// nesting, the paragraphs in other scripts, the catalogue, the blocks, and the -// closing paragraph with everything it closes after it. +// nesting, the paragraphs in other scripts, the catalogue, the links, the +// pictures, the blocks, and what closes the levels, the body and the document. // The markup itself - the skeleton, the label, the levels and the blocks - is // spelled in markup.go, in every layout and every way of carrying styles. @@ -19,6 +17,11 @@ const emptyTitle = `` // bytes so no escape in this file can be turned into something else on the way. var byteOrderMark = string([]byte{0xEF, 0xBB, 0xBF}) +// writeChunk is how much of what a page repeats is built before it is written, +// and so how often an interrupted run is noticed - the size the blocks are +// written in (core.FillRecords). +const writeChunk = 32 * 1024 + // prologue is everything in front of the label: the mark, and the skeleton down // to the body with whatever the catalogue and the style sheet put in the head - // or for a fragment the mark and the sheet. A fragment has no head, and a sheet @@ -32,56 +35,66 @@ func (s settings) prologue() string { if s.shape == structureDocument { p = m[headOpen] + s.catalogueHeadFor() + m[sheet] + m[headClose] } + if s.indented() { + p = joined(layOut(p, 0)) + } if s.bom { p = byteOrderMark + p } return p } -// opening is what follows the label: the levels of nesting, then the paragraphs -// in other scripts and the catalogue. Built when the file is written and never -// kept in the plan - a hundred thousand levels are 600 KB, and the engine holds -// the plan of every file at once. -func (s settings) opening() string { - return strings.Repeat(s.spelling()[divOpen], s.nesting) + s.internationalBlocks() + s.catalogueBlocks() +// depth is how many elements the content sits in: the levels of nesting, and +// for a page the html and body elements around them. The label sits at the +// depth of the body, outside the levels. +func (s settings) depth() int { + return s.bodyDepth() + s.nesting } -// openingBytes is the length of opening, worked out without building it. -func (s settings) openingBytes() int64 { - return int64(s.nesting*len(s.spelling()[divOpen]) + len(s.internationalBlocks()) + len(s.catalogueBlocks())) +// bodyDepth is the depth of what the body holds - none for a fragment, which +// has no body. +func (s settings) bodyDepth() int { + if s.shape == structureDocument { + return 2 + } + return 0 } -// blocks is the body builder for these settings. -// -// The settings have to reach the BUILDER and not only the prologue, and that is -// the part of this easy to miss: the bytes that close the levels, the body and -// the document sit in the last RECORD rather than in a footer. A change that -// swapped only the prologue would end a fragment with , or a page -// nested a thousand deep with not one - the right size, deterministic, -// and nonsense. -func (s settings) blocks() blocks { - m := s.spelling() - tail := m[paraClose] + strings.Repeat(m[divClose], s.nesting) - if s.shape == structureDocument { - tail += m[bodyClose] +// indents is the indentation the content is written with, and the one of what +// sits a level inside it - a list's items and a table's rows. Both are empty +// unless the page is indented, so the same writing serves every layout. +type indents struct{ lead, inner string } + +// indents builds them, when the file is written. A hundred thousand levels +// deep the lead is 200 KB, and the plan of every file is held at once, so it +// is never kept in one. +func (s settings) indents() indents { + if !s.indented() { + return indents{} } - return blocks{tail: tail, m: m} + return indents{lead: indentation(s.depth()), inner: indentation(s.depth() + 1)} } -// tailBytes is the length of what the last record ends with, worked out -// without building it. -func (s settings) tailBytes() int64 { - m := s.spelling() - n := len(m[paraClose]) + s.nesting*len(m[divClose]) - if s.shape == structureDocument { - n += len(m[bodyClose]) +// leadBytes and innerBytes are their lengths, worked out without building them. +func (s settings) leadBytes() int64 { + if !s.indented() { + return 0 + } + return indentationBytes(s.depth()) +} + +func (s settings) innerBytes() int64 { + if !s.indented() { + return 0 } - return int64(n) + return indentationBytes(s.depth() + 1) } // adds says the settings put something into the page beyond the skeleton and // the blocks, which is what the refusal of a size too small has to name. A page -// on one line is smaller than the same page on many, so formatting adds nothing. +// on one line is smaller than the same page on many, so only indented of the +// layouts adds anything. func (s settings) adds() bool { - return s.nesting > 0 || s.international || s.every || s.bom || s.styleName() != stylesNone + return s.nesting > 0 || s.international || s.every || s.bom || s.styleName() != stylesNone || + s.links > 0 || s.indented() } diff --git a/internal/format/htmlfile/pictures.go b/internal/format/htmlfile/pictures.go index 5e23ebc6..b1bdf175 100644 --- a/internal/format/htmlfile/pictures.go +++ b/internal/format/htmlfile/pictures.go @@ -46,23 +46,25 @@ func pictureProperties() []format.Property { } // pictureLine is one picture, planned: the file, and the line it sits on up -// to where the base64 starts and after it ends. +// to where the base64 starts and after it ends. An indented page writes the +// depth of its content in front of the line, which the plan does not keep. type pictureLine struct { desc format.Descriptor plan format.Plan open, close string } -// bytes is what the line takes in the page. -func (p pictureLine) bytes() int64 { - return int64(len(p.open)) + base64Length(p.plan.Bytes) + int64(len(p.close)) +// bytes is what the line takes in the page, its indentation lead bytes long. +func (p pictureLine) bytes(lead int64) int64 { + return lead + int64(len(p.open)) + base64Length(p.plan.Bytes) + int64(len(p.close)) } // planPictures plans the pictures, each with the seed of its place (RC4), so // asking for one more changes none of the others. The format is asked whether // it is a picture even when none are asked for: a value the setting cannot // take is a mistake either way, and eml refuses it the same way. -func planPictures(ctx context.Context, r format.Request, s picture.Settings, m *markup) ([]pictureLine, int64, error) { +func planPictures(ctx context.Context, r format.Request, page settings) ([]pictureLine, int64, error) { + s, m := page.pictures, page.spelling() desc, ok, err := picture.Of(s.Format) if err != nil { return nil, 0, err @@ -86,7 +88,7 @@ func planPictures(ctx context.Context, r format.Request, s picture.Settings, m * return nil, 0, pageTooLarge(cp.Bytes) } line := pictureLine{desc: desc, plan: cp, open: pictureOpen(m, desc, i+1), close: m[pictureTail]} - if sum += line.bytes(); sum > maxPage { + if sum += line.bytes(page.leadBytes()); sum > maxPage { return nil, 0, pageTooLarge(sum) } out = append(out, line) @@ -130,8 +132,11 @@ func describePictures(props map[string]any, s picture.Settings, lines []pictureL // writePictures writes every picture's line, each file made by its own // generator as it is written, so a picture of a hundred megabytes never sits in // memory. -func writePictures(ctx context.Context, w io.Writer, lines []pictureLine) error { +func writePictures(ctx context.Context, w io.Writer, lines []pictureLine, lead string) error { for _, p := range lines { + if err := core.WriteAll(w, []byte(lead)); err != nil { + return err + } if err := core.WriteAll(w, []byte(p.open)); err != nil { return err } diff --git a/internal/format/htmlfile/settings.go b/internal/format/htmlfile/settings.go index ee5c69d0..944de04e 100644 --- a/internal/format/htmlfile/settings.go +++ b/internal/format/htmlfile/settings.go @@ -43,6 +43,14 @@ const ( settingFormatting = "formatting" formattingLines = "lines" formattingMinified = "minified" + formattingIndented = "indented" + + settingLinks = "links" + // maxLinks is a list of about 70 MB, past the one threshold measured on + // 2026-10-14: DOMPurify sanitized a hundred thousand links in 4.7 s and + // ran out of an 8 GB heap over a million, which lxml read in 2.9 s and + // jsdom in 31 s (docs/HTML-5-2026-10-14.md section 10). + maxLinks = 1000000 settingStyles = "styles" stylesNone = "none" @@ -60,10 +68,12 @@ type settings struct { international, every bool // bom is a byte order mark in front of everything else. bom bool - // minified puts the whole page on one line, and styles says where its + // links is how many links the list carries, none when it is 0. + links int + // formatting is how the page is laid out, and styles says where its // style rules are, if anywhere. - minified bool - styles string + formatting string + styles string // pictures are the pictures written into the page. pictures picture.Settings } @@ -96,13 +106,20 @@ func properties() []format.Property { Detail: "Which elements the page uses. basic is headings, paragraphs, lists, tables and quotes. " + "every adds a block that uses every element of the HTML standard at least once, all 22 kinds of input and small pictures and a sound inside the page, with no script and nothing fetched from a network. " + "A fragment has no head, so it leaves out the eight elements that only a whole page holds.", + }, { + Name: settingLinks, Kind: format.PropertyInt, + Min: 0, Max: maxLinks, + Default: "0", + Detail: linksDetail, }, textenc.Axes(textenc.SettingBOM)[0], { Name: settingFormatting, Kind: format.PropertyChoice, - Choices: []string{formattingLines, formattingMinified}, + Choices: []string{formattingLines, formattingMinified, formattingIndented}, Default: formattingLines, - Detail: "How the page is laid out. minified puts the whole file on one line and ends without a newline, and it shows the same as lines. " + - "A program reading line by line gets the whole page as one line - Go's bufio.Scanner refuses a line past 64 KB unless given a larger buffer - " + - "and text read by joining what the elements hold runs the last word of one block into the first word of the next.", + Detail: "How the page is laid out. minified puts the whole file on one line and ends without a newline. " + + "indented puts two spaces in front of a line for every element it sits in, and leaves the text of pre as it is. All three show the same. " + + "A program reading line by line gets a minified page as one line - Go's bufio.Scanner refuses a line past 64 KB unless given a larger buffer - " + + "and text read by joining what the elements hold runs the last word of one block into the first word of the next. " + + "Indented with nesting puts more spaces in front of every line with each level, so nesting 1000 makes a page of at least 1.9 MB.", }, { Name: settingStyles, Kind: format.PropertyChoice, Choices: []string{stylesNone, stylesEmbedded, stylesInline}, @@ -147,12 +164,13 @@ func settingsOf(props map[string]string) (settings, error) { core.A("Basic", core.Choice{Of: settingElements, Value: elementsBasic}), core.A("Every", core.Choice{Of: settingElements, Value: elementsEvery}))) } - if v, ok := choice(props, settingFormatting, formattingLines, formattingMinified); ok { - s.minified = v == formattingMinified + if v, ok := choice(props, settingFormatting, formattingLines, formattingMinified, formattingIndented); ok { + s.formatting = v } else { - return settings{}, refused(props, settingFormatting, core.Says("html.HasToBeFormatting", "it has to be %s or %s", + return settings{}, refused(props, settingFormatting, core.Says("html.HasToBeFormatting", "it has to be %s, %s or %s", core.A("Lines", core.Choice{Of: settingFormatting, Value: formattingLines}), - core.A("Minified", core.Choice{Of: settingFormatting, Value: formattingMinified}))) + core.A("Minified", core.Choice{Of: settingFormatting, Value: formattingMinified}), + core.A("Indented", core.Choice{Of: settingFormatting, Value: formattingIndented}))) } if v, ok := choice(props, settingStyles, stylesNone, stylesEmbedded, stylesInline); ok { s.styles = v @@ -163,7 +181,12 @@ func settingsOf(props map[string]string) (settings, error) { core.A("Inline", core.Choice{Of: settingStyles, Value: stylesInline}))) } var err error - if s.nesting, err = nestingOf(props); err != nil { + if s.nesting, err = wholeNumber(props, settingNesting, maxNesting, + core.Says("html.HasToBeNesting", "it has to be a whole number from 0 to %d", core.A("Max", maxNesting))); err != nil { + return settings{}, err + } + if s.links, err = wholeNumber(props, settingLinks, maxLinks, + core.Says("html.HasToBeLinks", "it has to be a whole number from 0 to %d", core.A("Max", maxLinks))); err != nil { return settings{}, err } codec, err := textenc.Parse("html", props) @@ -192,22 +215,34 @@ func refused(props map[string]string, key string, reason core.Said) error { return &format.PropertyValueError{Format: "html", Key: key, Value: props[key], Reason: reason} } -// nestingOf reads how deep the content sits. -func nestingOf(props map[string]string) (int, error) { - v, ok := props[settingNesting] +// wholeNumber reads a setting that is a count from 0 to max, 0 when it is not +// set. The sentence comes from the caller, for the reason refused gives. +func wholeNumber(props map[string]string, key string, max int, reason core.Said) (int, error) { + v, ok := props[key] if !ok || v == "" { return 0, nil } n, err := strconv.Atoi(v) - if err != nil || n < 0 || n > maxNesting { - return 0, &format.PropertyValueError{ - Format: "html", Key: settingNesting, Value: v, - Reason: core.Says("html.HasToBeNesting", "it has to be a whole number from 0 to %d", core.A("Max", maxNesting)), - } + if err != nil || n < 0 || n > max { + return 0, &format.PropertyValueError{Format: "html", Key: key, Value: v, Reason: reason} } return n, nil } +// minified says the page is on one line, and indented that every line of it +// is indented by the elements it sits in. +func (s settings) minified() bool { return s.formatting == formattingMinified } +func (s settings) indented() bool { return s.formatting == formattingIndented } + +// formattingName is the value of formatting, lines for settings made without +// one. +func (s settings) formattingName() string { + if s.formatting == "" { + return formattingLines + } + return s.formatting +} + // manifest is what the manifest records about the settings, every one of them // whatever was asked, so a reader never has to know a default. The pictures // are recorded by describePictures, beside what they came to. @@ -219,17 +254,14 @@ func (s settings) manifest() map[string]any { if s.every { elements = elementsEvery } - layout := formattingLines - if s.minified { - layout = formattingMinified - } return map[string]any{ settingStructure: s.shape, settingNesting: s.nesting, settingCharacters: chars, settingElements: elements, + settingLinks: s.links, textenc.SettingBOM: s.bom, - settingFormatting: layout, + settingFormatting: s.formattingName(), settingStyles: s.styleName(), } } diff --git a/internal/guard/flagnames_test.go b/internal/guard/flagnames_test.go index 3a80f7a2..92e48f89 100644 --- a/internal/guard/flagnames_test.go +++ b/internal/guard/flagnames_test.go @@ -77,11 +77,58 @@ func packagesBelowTheSurfaces(t *testing.T) []string { return out } +// flagLike is a dash pair followed by a letter. "|---|---|" in generated +// markdown is three dashes and does not match, and neither does a range or an +// em dash written as two. +var flagLike = regexp.MustCompile(`--[a-z]`) + +// spellsAFlag says whether a text holds a dash pair followed by a letter other +// than the "xn--" every international domain name opens a label with in ASCII +// (RFC 3490) - an html page links to one (docs/HTML-5-2026-10-14.md). +// +// Forgiven by where the two dashes stand, not by taking the prefix out of the +// text first: taken out, "xn---limit" became "-limit" and the flag in it was +// gone (review of #185). Two matches cannot overlap, because a match ends on a +// letter and the next one would have to start on a dash, so the matches found +// are every dash pair there is. +func spellsAFlag(value string) bool { + for _, at := range flagLike.FindAllStringIndex(value, -1) { + if !opensAnInternationalLabel(value, at[0]) { + return true + } + } + return false +} + +// opensAnInternationalLabel says whether the dash pair at i is the one of an +// "xn--" starting a label - at the start of the text, or after a character +// that cannot be part of a name. "max-xn--limit" is a name with a flag in it. +func opensAnInternationalLabel(value string, i int) bool { + if i < 2 || value[i-2:i] != "xn" { + return false + } + if i == 2 { + return true + } + c := value[i-3] + inName := c == '-' || '0' <= c && c <= '9' || 'a' <= c && c <= 'z' || 'A' <= c && c <= 'Z' + return !inName +} + func TestNoMessageBelowTheSurfacesIsWrittenInFlagSpelling(t *testing.T) { - // A dash pair followed by a letter. "|---|---|" in generated markdown is - // three dashes and does not match, and neither does a range or an em dash - // written as two. - flagLike := regexp.MustCompile(`--[a-z]`) + // The canary: what the reading must find and what it must leave alone, so + // a change to what is forgiven shows here rather than as a quiet pass. + for value, flag := range map[string]bool{ + "Raise --limit above it": true, + "https://xn--bcher-kva.example/": false, + "xn---limit": true, + "max-xn--limit": true, + "https://xn--bcher-kva.example/ or --limit": true, + } { + if spellsAFlag(value) != flag { + t.Fatalf("%q was read as a flag: %v. It is %v, so this guard reads text wrongly", value, !flag, flag) + } + } dirs := packagesBelowTheSurfaces(t) scanned, found := 0, 0 @@ -113,7 +160,7 @@ func TestNoMessageBelowTheSurfacesIsWrittenInFlagSpelling(t *testing.T) { if err != nil { return true } - if flagLike.MatchString(value) { + if spellsAFlag(value) { found++ t.Errorf("%s writes a setting in command line spelling:\n %q\n"+ "This text is shown by the window too, and there are no flags on it.", diff --git a/internal/guard/generatorbytes_test.go b/internal/guard/generatorbytes_test.go index f8ba1fbf..1f5a11cf 100644 --- a/internal/guard/generatorbytes_test.go +++ b/internal/guard/generatorbytes_test.go @@ -474,6 +474,14 @@ func pageGoldenCases() map[string]engine.Target { Properties: map[string]string{"formatting": "minified", "elements": "every", "styles": "embedded"}}, "html_8kib_minified_inline": {ID: "g", Format: "html", Sizes: engine.Uniform(1, 8192), Label: true, Properties: map[string]string{"formatting": "minified", "styles": "inline"}}, + // Levels of nesting on one line, in a page and in a fragment: what + // closes the levels and the document is the end of the file, and these + // hold it to the bytes it had while it sat in the last block. Measured + // with the binary of main before that moved (docs/HTML-5-2026-10-14.md). + "html_odd_minified_nested_embedded_international": {ID: "g", Format: "html", Sizes: engine.Uniform(1, 8193), Label: true, + Properties: map[string]string{"formatting": "minified", "nesting": "40", "styles": "embedded", "characters": "international"}}, + "html_8kib_fragment_minified_nested": {ID: "g", Format: "html", Sizes: engine.Uniform(1, 8192), Label: true, + Properties: map[string]string{"structure": "fragment", "formatting": "minified", "nesting": "3"}}, "html_odd_fragment_inline_international_pictures": {ID: "g", Format: "html", Sizes: engine.Uniform(1, 8193), Label: true, Properties: map[string]string{"structure": "fragment", "styles": "inline", "characters": "international", "nesting": "2", "inline": "1", "inline_size": "1kb"}}, diff --git a/internal/guard/htmllayout_test.go b/internal/guard/htmllayout_test.go new file mode 100644 index 00000000..f5288af8 --- /dev/null +++ b/internal/guard/htmllayout_test.go @@ -0,0 +1,319 @@ +package guard + +import ( + "bytes" + "context" + "errors" + "maps" + "os" + "path/filepath" + "regexp" + "strconv" + "strings" + "testing" + + "github.com/donislawdev/TestingFilesGenerator/internal/format" + "github.com/donislawdev/TestingFilesGenerator/internal/oracle" +) + +// An indented page and a page of links, both 2026-10-14 +// (docs/HTML-5-2026-10-14.md). Every guard here reads the BYTES, for the reason +// htmlfragment_test.go gives, and the indentation and the kinds of link are +// worked out here from what the settings mean rather than taken from the +// generator. + +// htmlIndentCombos are the pages of lines the indentation guard indents: each +// shape, the levels, the catalogue with its pre, the paragraphs in other +// scripts behind a mark, both ways of carrying styles, pictures and links. +var htmlIndentCombos = []map[string]string{ + {}, + {"structure": "fragment"}, + {"nesting": "3"}, + {"nesting": "40", "structure": "fragment"}, + {"elements": "every"}, + {"elements": "every", "structure": "fragment", "nesting": "2"}, + {"characters": "international", "bom": "true"}, + {"styles": "embedded"}, + {"styles": "embedded", "structure": "fragment", "bom": "true"}, + {"styles": "inline", "characters": "international", "links": "20"}, + {"inline": "2", "inline_size": "1kb", "nesting": "2"}, + {"links": "40", "elements": "every", "nesting": "5", "styles": "inline"}, +} + +// htmlVoid are the elements of HTML with no end tag, from the standard. +var htmlVoid = map[string]bool{"area": true, "base": true, "br": true, "col": true, "embed": true, "hr": true, + "img": true, "input": true, "link": true, "meta": true, "source": true, "track": true, "wbr": true} + +// htmlIndentByElements is a page of lines indented the way the setting says: +// two spaces in front of a line for every element open where it starts, one +// fewer when the line starts by closing one, and nothing in front of the text +// of pre. Written here and not borrowed, so the generator and this cannot +// agree on a wrong depth. +func htmlIndentByElements(page []byte) []byte { + var out bytes.Buffer + depth, inPre := 0, false + for _, line := range strings.SplitAfter(string(page), "\n") { + if line == "" { + continue + } + if !inPre { + lead := depth + if strings.HasPrefix(strings.TrimPrefix(line, string([]byte{0xEF, 0xBB, 0xBF})), "") { + continue + } + if name == "pre" { + inPre = m[1] == "" + } + if m[1] == "" { + depth++ + } else { + depth-- + } + } + } + return out.Bytes() +} + +// writeUnlabelled is writeEncoded without the label, whose text carries the +// size and so differs between two files of different sizes. +func writeUnlabelled(t *testing.T, size int64, props map[string]string) []byte { + t.Helper() + d, err := format.Get("html") + if err != nil { + t.Fatal(err) + } + p, err := d.Generator.Plan(format.Request{Bytes: size, Seed: 7741, Properties: props}) + if err != nil { + t.Fatalf("planning %d B with %v: %v", size, props, err) + } + var buf bytes.Buffer + if err := d.Generator.Write(context.Background(), &buf, p); err != nil { + t.Fatalf("writing %d B with %v: %v", size, props, err) + } + if int64(buf.Len()) != size { + t.Fatalf("%v: ordered %d B and produced %d", props, size, buf.Len()) + } + return buf.Bytes() +} + +// TestAnIndentedPageIsThePageOfLinesIndentedByItsElements is the claim of +// formatting=indented, to the byte. +// +// The page of lines is indented here, and the indented page of exactly that +// size with the same seed has to be those bytes. They are the same blocks: an +// indented block is its block of lines and its indentation, so a block that +// fits one fits the other, and the closing paragraph is left the same number +// of bytes. It ASKS that the indentation did something, so a setting that +// stopped reaching the writer cannot pass as a page with nothing to indent. +func TestAnIndentedPageIsThePageOfLinesIndentedByItsElements(t *testing.T) { + for _, props := range htmlIndentCombos { + lines := writeUnlabelled(t, 16384, props) + want := htmlIndentByElements(lines) + if bytes.Equal(want, lines) { + t.Fatalf("%v: indenting the page of lines changed nothing, so this compares nothing", props) + } + indented := maps.Clone(props) + indented["formatting"] = "indented" + got := writeUnlabelled(t, int64(len(want)), indented) + if !bytes.Equal(got, want) { + gl, wl := strings.Split(string(got), "\n"), strings.Split(string(want), "\n") + at := 0 + for at < len(gl) && at < len(wl) && gl[at] == wl[at] { + at++ + } + t.Errorf("%v: the indented page is not the page of lines indented by its elements - they part at line %d", props, at+1) + } + } +} + +// TestTheLabelOfAnIndentedPageSitsInTheBody holds the one line the comparison +// above leaves out: the label stands at the depth of the body, outside the +// levels, so a reader that drops the deep ones keeps it. +func TestTheLabelOfAnIndentedPageSitsInTheBody(t *testing.T) { + for _, c := range []struct { + props map[string]string + lead string + }{ + {map[string]string{"formatting": "indented", "nesting": "4"}, "\n

    tfg"}, + {map[string]string{"formatting": "indented", "nesting": "4", "structure": "fragment"}, "

    tfg"}, + } { + body := writeEncoded(t, "html", 16384, c.props) + at := bytes.Index(body, []byte(c.lead)) + if at < 0 || (c.props["structure"] == "fragment" && at != 0) { + t.Errorf("%v: the label does not stand where the body puts it (%q)", c.props, c.lead) + } + } +} + +var ( + // htmlLinkList is the list of links, its items and the address of each. + htmlLinkList = regexp.MustCompile(`(?s)\n`) + htmlLinkHref = regexp.MustCompile(`
  • ]*>([^<]+)
    • \n`) + // htmlLinkHost is the host of an address that names one. + htmlLinkHost = regexp.MustCompile(`^(?:[a-z]+:)?//([^/:?#]+)`) +) + +// TestTheLinksAreTheKindsInTurnOnDomainsKeptForExamples is the claim of links. +// +// As many links as were ordered, each kind once before any repeats, in the +// same turn after it, the first pointing at the list itself - which is there +// to point at - and no address naming a host outside the domains RFC 2606 keeps +// for examples. The list sits above the pictures, which a reader can stop +// short of (docs/HTML-3-2026-10-14.md section 9). +func TestTheLinksAreTheKindsInTurnOnDomainsKeptForExamples(t *testing.T) { + const kinds = 18 + for _, n := range []int{0, 1, kinds, kinds + 1, 2*kinds + 9} { + for _, shape := range []string{"document", "fragment"} { + props := map[string]string{"links": strconv.Itoa(n), "structure": shape, "inline": "1", "inline_size": "1kb"} + body := writeEncoded(t, "html", 32768, props) + list := htmlLinkList.FindSubmatchIndex(body) + if n == 0 { + if list != nil || bytes.Contains(body, []byte("tfg-links")) { + t.Errorf("%v: no link was ordered and the page carries the list", props) + } + continue + } + if list == nil { + t.Fatalf("%v: the page has no list of links", props) + } + if pic := bytes.Index(body, []byte(`inline_`)); pic < list[1] { + t.Errorf(= kinds && href != string(items[i-kinds][1]) { + t.Errorf("%v: link %d is %s, and the kinds in turn make %s", props, i+1, href, items[i-kinds][1]) + } + if h := htmlLinkHost.FindStringSubmatch(href); h != nil && + h[1] != "example.com" && h[1] != "example.net" && h[1] != "example.org" && !strings.HasSuffix(h[1], ".example") { + t.Errorf("%v: link %d names %s, which is not a domain kept for examples", props, i+1, h[1]) + } + } + } + } +} + +// TestTheStructuralCheckRefusesAListOfLinksThatIsMoreThanItsLinks holds the +// hole the review of #185 led to in the checker, with a verdict for each. +// +// The checker counted the links between the list and the next closing tag. A +// list closed only at the end of the page carried every block inside it, and a +// line that was no link sat between two that were, and both counted right. In +// a page of lines or on one line nothing else saw it - the tags still balance, +// and only an indented page has its indentation read. Every page here is one +// the program wrote, broken one way. +func TestTheStructuralCheckRefusesAListOfLinksThatIsMoreThanItsLinks(t *testing.T) { + const holds = "links=5 was ordered and the list of links holds something other than links" + type broken struct { + props map[string]string + spoil func(page []byte, nl string) []byte + verdict string + } + head := []byte(`")) + out := append(append([]byte{}, b[:end]...), b[end+len(""+nl):]...) + if at := bytes.LastIndex(out, []byte("")); at >= 0 { + return append(append(append([]byte{}, out[:at]...), ""+nl...), out[at:]...) + } + return append(out, ""+nl...) + } + notALink := func(b []byte, nl string) []byte { + at := bytes.Index(b, head) + len(head) + len(nl) + return append(append(append([]byte{}, b[:at]...), "
  • not a link
    • "+nl...), b[at:]...) + } + neverClosed := func(b []byte, _ string) []byte { + list := bytes.Index(b, head) + return append(append([]byte{}, b[:list]...), bytes.ReplaceAll(b[list:], []byte(""), nil)...) + } + var cases []broken + for _, layout := range []string{"lines", "minified"} { + page := map[string]string{"links": "5", "formatting": layout} + fragment := map[string]string{"links": "5", "formatting": layout, "structure": "fragment"} + cases = append(cases, + broken{page, closedAtTheEnd, holds}, + broken{fragment, closedAtTheEnd, holds}, + broken{page, notALink, holds}, + broken{page, neverClosed, "links=5 was ordered and the list of links is never closed"}) + } + dir := t.TempDir() + for i, c := range cases { + nl := "\n" + if c.props["formatting"] == "minified" { + nl = "" + } + page := writeEncoded(t, "html", 16384, c.props) + bad := c.spoil(page, nl) + if bytes.Equal(bad, page) { + t.Fatalf("%v: breaking the page changed nothing, so there is nothing to refuse", c.props) + } + path := filepath.Join(dir, strconv.Itoa(i)+".html") + if err := os.WriteFile(path, bad, 0o600); err != nil { + t.Fatal(err) + } + res := oracle.Strict("html", path, htmlSettingWords(c.props)...) + if !res.Available { + t.Skip("the structural check needs python") + } + if res.Err == nil || !strings.HasPrefix(res.Err.Error(), c.verdict) { + t.Errorf("%v, case %d: a page broken on purpose was answered with %v, and %q was the verdict owed", c.props, i, res.Err, c.verdict) + } + } +} + +// TestADeepIndentedPageAndAMillionLinksStopSoonAfterARunIsStopped holds the +// two things a page now writes a chunk at a time rather than at once: a +// hundred thousand indented levels, 10 GB in front of the first block, and a +// million links. Counted in bytes rather than timed, for the reason +// picturestop_test.go gives. +func TestADeepIndentedPageAndAMillionLinksStopSoonAfterARunIsStopped(t *testing.T) { + const stopAt = int64(1 << 20) + d, err := format.Get("html") + if err != nil { + t.Fatal(err) + } + for _, props := range []map[string]string{ + {"nesting": "100000", "formatting": "indented"}, + {"links": "1000000"}, + } { + size := d.SmallestAccepted(format.Request{Seed: 7741, Label: true, Properties: props}) + if size < 64*stopAt { + t.Fatalf("%v: the smallest page is %d B, too small to ask whether it stops early", props, size) + } + p, err := d.Generator.Plan(format.Request{Bytes: size, Seed: 7741, Label: true, Properties: props}) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + w := &stoppingWriter{cancel: cancel, after: stopAt} + err = d.Generator.Write(ctx, w, p) + cancel() + if !errors.Is(err, context.Canceled) { + t.Errorf("%v: stopped after 1 MiB of %d B and the write ended with %v, not the stop", props, size, err) + } + if w.n > 4*stopAt { + t.Errorf("%v: stopped after 1 MiB and wrote %d B of %d B before it noticed", props, w.n, size) + } + } +} diff --git a/internal/guard/htmlsettings_test.go b/internal/guard/htmlsettings_test.go index 0abc3dc4..7f876bfc 100644 --- a/internal/guard/htmlsettings_test.go +++ b/internal/guard/htmlsettings_test.go @@ -44,6 +44,12 @@ var htmlCombos = []map[string]string{ {"styles": "inline", "characters": "international"}, {"formatting": "minified", "styles": "inline", "elements": "every", "inline": "1", "inline_size": "1kb", "nesting": "2"}, {"formatting": "minified", "styles": "embedded", "elements": "every", "structure": "fragment"}, + {"links": "1"}, + {"links": "40", "structure": "fragment", "nesting": "2"}, + {"formatting": "indented"}, + {"formatting": "indented", "structure": "fragment", "nesting": "3", "bom": "true"}, + {"formatting": "indented", "styles": "inline", "elements": "every", "links": "20", "inline": "1", "inline_size": "1kb", "nesting": "2"}, + {"formatting": "indented", "styles": "embedded", "characters": "international", "structure": "fragment"}, } // htmlLineEnd is what ends a line of a page of these settings - nothing, when @@ -55,27 +61,53 @@ func htmlLineEnd(props map[string]string) string { return "\n" } +// htmlBodyDepth is how many elements the content of a page sits in before the +// levels: html and body, or none in a fragment. +func htmlBodyDepth(props map[string]string) int { + if props["structure"] == "fragment" { + return 0 + } + return 2 +} + +// htmlIndent is the indentation of a line depth levels deep on a page of +// these settings - two spaces a level when it is indented, nothing otherwise. +func htmlIndent(props map[string]string, depth int) string { + if props["formatting"] != "indented" { + return "" + } + return strings.Repeat(" ", depth) +} + // htmlClosing is what a file of these settings ends with after its closing // paragraph, written here from the shape of HTML rather than taken from the -// generator. +// generator: each level closed at the depth it opened at, then the body a +// level in and the document at the margin. func htmlClosing(props map[string]string) string { nl := htmlLineEnd(props) n, _ := strconv.Atoi(props["nesting"]) - end := strings.Repeat(""+nl, n) + end := "" + for k := n - 1; k >= 0; k-- { + end += htmlIndent(props, htmlBodyDepth(props)+k) + "" + nl + } if props["structure"] != "fragment" { - end += "" + nl + "" + nl + end += htmlIndent(props, 1) + "" + nl + "" + nl } return end } // htmlEmptyEnd matches the end of the smallest file of these settings: an -// empty closing paragraph, with the style it carries when styles are inline, -// and what closes after it. +// empty closing paragraph at the depth of the content, with the style it +// carries when styles are inline, and what closes after it. func htmlEmptyEnd(props map[string]string) *regexp.Regexp { open := `

    ` if props["styles"] == "inline" { open = `

    ` } + if props["formatting"] == "indented" { + n, _ := strconv.Atoi(props["nesting"]) + open = regexp.QuoteMeta("\n"+htmlIndent(props, htmlBodyDepth(props)+n)) + open + } return regexp.MustCompile(open + `

    ` + regexp.QuoteMeta(htmlLineEnd(props)+htmlClosing(props)) + `\z`) } @@ -169,8 +201,9 @@ func TestEveryHTMLSettingIsRecordedInTheManifest(t *testing.T) { want map[string]any }{ {nil, map[string]any{"structure": "document", "nesting": 0, "characters": "ascii", "elements": "basic", "bom": false, - "formatting": "lines", "styles": "none", "line_ending": "lf", + "formatting": "lines", "styles": "none", "line_ending": "lf", "links": 0, "inline": 0, "inline_format": "png", "inline_size": int64(8192), "inline_bytes": int64(0), "inline_encoded_bytes": int64(0)}}, + {map[string]string{"formatting": "indented", "links": "25"}, map[string]any{"formatting": "indented", "line_ending": "lf", "links": 25}}, {map[string]string{"structure": "fragment", "nesting": "7", "characters": "international", "elements": "every", "bom": "true", "formatting": "minified", "styles": "inline", "inline": "2", "inline_format": "gif", "inline_size": "1kb"}, map[string]any{"structure": "fragment", "nesting": 7, "characters": "international", "elements": "every", "bom": true, @@ -195,7 +228,7 @@ func TestEveryHTMLSettingIsRecordedInTheManifest(t *testing.T) { // routes through the parser and have to meet. func TestTheDefaultHTMLSettingsAreTheBytesItAlwaysWrote(t *testing.T) { spoken := map[string]string{"nesting": "0", "characters": "ascii", "elements": "basic", "bom": "false", - "formatting": "lines", "styles": "none", "inline": "0", "inline_format": "png", "inline_size": "8kb"} + "formatting": "lines", "styles": "none", "links": "0", "inline": "0", "inline_format": "png", "inline_size": "8kb"} for _, size := range []int64{118, 119, 4096} { if !bytes.Equal(writeEncoded(t, "html", size, nil), writeEncoded(t, "html", size, spoken)) { t.Errorf("at %d B: saying nothing and saying every default produce different bytes", size) @@ -233,6 +266,15 @@ func TestTheStructuralCheckIsToldEveryHTMLSetting(t *testing.T) { // between them - the honest half of this row is what keeps the checker // reading every rule there (review of #179). {map[string]string{"styles": "embedded", "formatting": "minified"}, map[string]string{"styles": "inline", "formatting": "minified"}}, + // An indented page told it is lines, and the other way round: the + // checker reads every line's indentation, so either is a line it + // refuses. And links counted to the one, not taken from the page. + {map[string]string{"formatting": "indented", "nesting": "2"}, map[string]string{"nesting": "2"}}, + {map[string]string{"nesting": "2"}, map[string]string{"formatting": "indented", "nesting": "2"}}, + {map[string]string{"formatting": "indented", "elements": "every"}, map[string]string{"elements": "every"}}, + {map[string]string{"links": "19"}, map[string]string{"links": "18"}}, + {map[string]string{"links": "19"}, map[string]string{}}, + {map[string]string{}, map[string]string{"links": "1"}}, } { path := filepath.Join(dir, strconv.Itoa(i)+".html") if err := os.WriteFile(path, writeEncoded(t, "html", 32768, c.made), 0o600); err != nil { diff --git a/internal/guard/mutationcoverage_test.go b/internal/guard/mutationcoverage_test.go index c0e93d31..86d40a77 100644 --- a/internal/guard/mutationcoverage_test.go +++ b/internal/guard/mutationcoverage_test.go @@ -228,7 +228,10 @@ var provenByProbe = map[string]string{ "a number removed from the middle of a range, and the summary table in CLAUDE.md announcing a range the document no longer defines. All three went red.", "TestEveryGuardIsEitherProvenByMutationOrListedAsNotProven": "checked 2026-08-01 three ways: a guard added with neither a mutation nor an entry, " + - "an entry naming a guard that was renamed away, and a guard left on the list after it gained a mutation. All three went red.", + "an entry naming a guard that was renamed away, and a guard left on the list after it gained a mutation. All three went red. " + + "And 2026-10-14 (review of #185): the name read between two quote classes again, with an empty group kept for the second " + + "capture. Red with the canary's own message. Not a mutation entry, because on the machine with tools/ this guard was " + + "red that day before anything was broken, ten old findings, so a runner would have called any entry caught.", "TestEveryLinkBetweenDocumentsLeadsSomewhere": "checked 2026-08-01 by adding a link to a document that does not exist. It went red.", @@ -249,10 +252,33 @@ var provenByProbe = map[string]string{ var ( testDeclaration = regexp.MustCompile(`(?m)^func (Test\w+)\(`) - mutationTarget = regexp.MustCompile(`"(Test\w+)"`) + // Either quote: Python writes a string in single quotes as readily as in + // double ones, and an entry written by repr() comes out in single. With + // only the double quote read, nine of the guards this reported as unproven + // on 2026-10-14 had entries it could not see (O280). A pair of the same + // quote, each in its own alternative: two classes side by side also read + // 'TestFoo" out of the middle of a longer string (review of #185). + mutationTarget = regexp.MustCompile(`"(Test\w+)"|'(Test\w+)'`) ) +// mutationTargets is every guard name the mutation list holds as a whole +// string, in either quote. +func mutationTargets(body string) map[string]bool { + named := map[string]bool{} + for _, m := range mutationTarget.FindAllStringSubmatch(body, -1) { + named[m[1]+m[2]] = true + } + return named +} + func TestEveryGuardIsEitherProvenByMutationOrListedAsNotProven(t *testing.T) { + // The canary, before the list is looked for, so a fresh checkout runs it + // too: both quotes read, a quote of one kind closed by the other not. + sample := `("a", "f.go", "x", "y", "TestDouble"), ('b', 'f.go', 'x', 'y', 'TestSingle'), "'TestMixed"` + if got := mutationTargets(sample); len(got) != 2 || !got["TestDouble"] || !got["TestSingle"] { + t.Fatalf("the mutation list is read wrongly: %v from %s, and TestDouble and TestSingle alone were in it", got, sample) + } + root := repoRoot(t) // The mutation list lives in tools/, which is excluded from the repository, @@ -265,10 +291,7 @@ func TestEveryGuardIsEitherProvenByMutationOrListedAsNotProven(t *testing.T) { return } - named := map[string]bool{} - for _, m := range mutationTarget.FindAllStringSubmatch(string(body), -1) { - named[m[1]] = true - } + named := mutationTargets(string(body)) if len(named) == 0 { t.Fatal("the mutation list names no test at all - this guard would pass without checking anything") } diff --git a/internal/guard/notelemetry_test.go b/internal/guard/notelemetry_test.go index 64874492..c1c53148 100644 --- a/internal/guard/notelemetry_test.go +++ b/internal/guard/notelemetry_test.go @@ -134,11 +134,12 @@ func ourPackage(p string) (string, bool) { // allowedURLs are the addresses that may appear as literals in shipped code. // -// Every one of them but the last two is an XML namespace, which is an -// identifier and not an address: it names a vocabulary, it is written INTO the -// files this tool generates, and nothing ever fetches it. They are here anyway, -// because a scan that guessed which URLs are "only identifiers" would be -// guessing at exactly the shape somebody would use to hide one. +// Most of them are XML namespaces, which are identifiers and not addresses: +// a namespace names a vocabulary, it is written INTO the files this tool +// generates, and nothing ever fetches it. The links of an html page are +// written into the page the same way. They are here anyway, because a scan +// that guessed which URLs are "only identifiers" would be guessing at exactly +// the shape somebody would use to hide one. var allowedURLs = map[string]string{ "internal/format/docx/docx.go|http://schemas.openxmlformats.org/officeDocument/2006/relationships": "OPC namespace, written into the document", "internal/format/docx/docx.go|http://schemas.openxmlformats.org/wordprocessingml/2006/main": "WordprocessingML namespace", @@ -155,6 +156,18 @@ var allowedURLs = map[string]string{ "internal/version/version.go|https://www.gnu.org/licenses/gpl-3.0.html": "where the GPL text is, named by the licence notice this tool prints", "internal/gui/text/screens.go|https://donislawdev.com/support/": "the support page, handed to the system browser on a click - " + "untouchable rule 8 permits this and permits nothing else like it", + // The links an html page carries when links is set, written INTO the page + // on domains RFC 2606 keeps for examples, and never followed by anything in + // this program (docs/HTML-5-2026-10-14.md section 2.1). + "internal/format/htmlfile/links.go|https://example.com/path/page.html?a=1&b=2#part": "a link written into a page, on a domain kept for examples", + "internal/format/htmlfile/links.go|http://example.org/": "a link written into a page, on a domain kept for examples", + "internal/format/htmlfile/links.go|https://example.com/caf%C3%A9?q=two%20words": "a link written into a page, on a domain kept for examples", + "internal/format/htmlfile/links.go|https://xn--bcher-kva.example/": "a link written into a page, on a top level domain kept for examples", + "internal/format/htmlfile/links.go|https://example.com:8443/port": "a link written into a page, on a domain kept for examples", + "internal/format/htmlfile/links.go|ftp://example.com/file.txt": "a link written into a page, on a domain kept for examples", + "internal/format/htmlfile/links.go|https://example.com/files/report.pdf": "a link written into a page, on a domain kept for examples", + "internal/format/htmlfile/links.go|https://example.org/new": "a link written into a page, on a domain kept for examples", + "internal/format/htmlfile/links.go|https://example.net/sponsored": "a link written into a page, on a domain kept for examples", "internal/legal/companions.go|https://github.com/pal1000/mesa-dist-win/releases/download/26.2.0/mesa3d-26.2.0-release-msvc.7z": "where the software " + "renderer's bytes were downloaded FROM, by the release workflow and never by the program: written into the bill of " + "materials as the package's download location, and compared with the workflow's pin by a guard", diff --git a/internal/guard/parity_test.go b/internal/guard/parity_test.go index 6e65a6e2..a1dfa4e9 100644 --- a/internal/guard/parity_test.go +++ b/internal/guard/parity_test.go @@ -201,6 +201,7 @@ var reachableFromTheWindow = []string{ "property:html.inline", "property:html.inline_format", "property:html.inline_size", + "property:html.links", "property:html.nesting", "property:html.structure", "property:html.styles", diff --git a/internal/guard/testdata/generator-golden.json b/internal/guard/testdata/generator-golden.json index ef4995c4..eb465915 100644 --- a/internal/guard/testdata/generator-golden.json +++ b/internal/guard/testdata/generator-golden.json @@ -77,6 +77,10 @@ "bytes": 8192, "sha256": "7fcd7bd8fa49fb5e7392ef33d8f1537c96dabb2157bbda7cfe2eba75c02513a7" }, + "html_8kib_fragment_minified_nested": { + "bytes": 8192, + "sha256": "9adc4dd5cfe3d34a349195e3dda5b94c7dc3de3a1bf2a8c8e676afd60492e53f" + }, "html_8kib_minified_inline": { "bytes": 8192, "sha256": "3cdeab67f740f775efa438a83ac45006b81ca46db81e850e885c180ecf926288" @@ -97,6 +101,10 @@ "bytes": 16385, "sha256": "c654ffeef0bd8f07115f647620c47ca12849d7a26614bf389682fe1e7e3ad5fd" }, + "html_odd_minified_nested_embedded_international": { + "bytes": 8193, + "sha256": "4026208c4fa88a9e4d73f4cc45c3d9343e4ceb2d4836861bee2d1794cb7ea73e" + }, "ico_32kib": { "bytes": 32768, "sha256": "da019bf067004ef7290aca46a5e6d68dd4d0c77bf419511a01166ff17e425f2b", diff --git a/internal/guard/testdata/screens/catalogue.png b/internal/guard/testdata/screens/catalogue.png index 132061d7..66e80b50 100644 Binary files a/internal/guard/testdata/screens/catalogue.png and b/internal/guard/testdata/screens/catalogue.png differ diff --git a/internal/guard/testdata/screens/catalogue.xml b/internal/guard/testdata/screens/catalogue.xml index 5834934e..9c04e26f 100644 --- a/internal/guard/testdata/screens/catalogue.xml +++ b/internal/guard/testdata/screens/catalogue.xml @@ -2559,13 +2559,13 @@ - - Bom - - + + BOM + + - + diff --git a/internal/gui/text/labels.go b/internal/gui/text/labels.go index 1c9adafc..65966eaf 100644 --- a/internal/gui/text/labels.go +++ b/internal/gui/text/labels.go @@ -40,12 +40,14 @@ func EnglishLabel(key string) string { // respelled are the keys whose spelling out is a word somebody meeting the // window for the first time does not know. Inline is what MIME calls a picture // the text shows (the owner's decision of 2026-10-14, -// docs/EML-2-2026-10-13.md section 8). The key is still one hover away. +// docs/EML-2-2026-10-13.md section 8). The key is still one hover away. bom is +// an abbreviation, and spelled out of the key it read "Bom" (O278). // // Keyed by the recipe key rather than declared on a format's setting, because // a label belongs to the key: every format, preset and tool declaring one key // shares one Label entry in the catalogue. var respelled = map[string]string{ + "bom": "BOM", "inline": "Pictures", "inline_format": "Picture format", "inline_size": "Picture size", diff --git a/internal/gui/text/locale/registry/en.json b/internal/gui/text/locale/registry/en.json index 1645402f..4b38c5e8 100644 --- a/internal/gui/text/locale/registry/en.json +++ b/internal/gui/text/locale/registry/en.json @@ -1331,8 +1331,8 @@ }, "Detail.format/html.formatting": { "description": "The sentence under the formatting setting of the html format, after what it takes.", - "hash": "sha256-4aa9c84f745f", - "other": "How the page is laid out. minified puts the whole file on one line and ends without a newline, and it shows the same as lines. A program reading line by line gets the whole page as one line - Go's bufio.Scanner refuses a line past 64 KB unless given a larger buffer - and text read by joining what the elements hold runs the last word of one block into the first word of the next." + "hash": "sha256-6d6322985c6f", + "other": "How the page is laid out. minified puts the whole file on one line and ends without a newline. indented puts two spaces in front of a line for every element it sits in, and leaves the text of pre as it is. All three show the same. A program reading line by line gets a minified page as one line - Go's bufio.Scanner refuses a line past 64 KB unless given a larger buffer - and text read by joining what the elements hold runs the last word of one block into the first word of the next. Indented with nesting puts more spaces in front of every line with each level, so nesting 1000 makes a page of at least 1.9 MB." }, "Detail.format/html.inline": { "description": "The sentence under the inline setting of the html format, after what it takes.", @@ -1349,6 +1349,11 @@ "hash": "sha256-2433610ca9da", "other": "How big each picture is. Inside the page it is base64 on one line, about a third larger. lxml, and BeautifulSoup on top of it, leave out a picture past about 7.1 MB unless told to read huge trees. On Linux lxml took over 20 seconds over such a page, and with a 15 MB picture it lost the rest of the page as well." }, + "Detail.format/html.links": { + "description": "The sentence under the links setting of the html format, after what it takes.", + "hash": "sha256-c07164812721", + "other": "How many links the page carries, in a list above the pictures and the blocks. They go through 18 kinds in turn and then repeat - a link to the list itself, relative, to the folder above, from the root, a query alone, absolute with a query and a fragment, http, without a scheme, percent encoded, an international domain, a port, mailto:, tel:, ftp:, a download, a new window, nofollow and data: - and the text of each says which it is. Every address is on a domain kept for examples. DOMPurify with its defaults takes out the data: address and the target that opens a new window, and it ran out of 8 GB of memory over a million links, which lxml read in 3 seconds. The catalogue of every element has six links of its own, which this does not count." + }, "Detail.format/html.nesting": { "description": "The sentence under the nesting setting of the html format, after what it takes.", "hash": "sha256-cac70df12b4b", @@ -2146,8 +2151,8 @@ }, "Label.bom": { "description": "The name beside the box of a setting a recipe writes as bom.", - "hash": "sha256-e39338101951", - "other": "Bom" + "hash": "sha256-1471d8c7a9cf", + "other": "BOM" }, "Label.bulk": { "description": "The name beside the box of a setting a recipe writes as bulk.", @@ -2409,6 +2414,11 @@ "hash": "sha256-c948fba2b0dd", "other": "Line endings" }, + "Label.links": { + "description": "The name beside the box of a setting a recipe writes as links.", + "hash": "sha256-9024c197d849", + "other": "Links" + }, "Label.message_depth": { "description": "The name beside the box of a setting a recipe writes as message_depth.", "hash": "sha256-8a18b09b1f37", diff --git a/internal/gui/text/locale/registry/pl.json b/internal/gui/text/locale/registry/pl.json index 4249756c..87156346 100644 --- a/internal/gui/text/locale/registry/pl.json +++ b/internal/gui/text/locale/registry/pl.json @@ -265,10 +265,11 @@ "Detail.format/html.bom": { "hash": "sha256-a2d6a88583d6", "other": "Czy plik zaczyna się od znacznika kolejności bajtów (BOM). Czytnik, który musi zgadywać kodowanie, go potrzebuje, a czytnik, który się go nie spodziewa, pokazuje go jako obce znaki na początku pliku." }, "Detail.format/html.characters": { "hash": "sha256-846c2063c0e0", "other": "Czy strona zawiera tekst spoza ASCII. międzynarodowe dokłada akapity po polsku, rosyjsku, japońsku i arabsku, arabski od prawej do lewej, z emoji, literą zapisaną dwoma punktami kodowymi oraz encjami nazwanymi i liczbowymi. Program bez fontu japońskiego albo arabskiego pokazuje w ich miejscu kwadraty." }, "Detail.format/html.elements": { "hash": "sha256-ad89039f3061", "other": "Których elementów używa strona. podstawowe to nagłówki, akapity, listy, tabele i cytaty. wszystkie dokłada blok, który używa każdego elementu standardu HTML co najmniej raz, wszystkich 22 rodzajów pola input oraz małych obrazków i dźwięku zapisanych w samej stronie, bez skryptu i bez niczego pobieranego z sieci. Fragment nie ma głowy, więc pomija osiem elementów, które ma tylko cała strona." }, - "Detail.format/html.formatting": { "hash": "sha256-4aa9c84f745f", "other": "Jak ułożona jest strona. zminifikowany stawia cały plik w jednej linii i kończy go bez znaku nowej linii, a strona wygląda tak samo jak w wierszach. Program czytający wierszami dostaje całą stronę jako jeden wiersz - bufio.Scanner w Go odmawia wiersza dłuższego niż 64 KB, chyba że dostanie większy bufor - a tekst czytany przez łączenie tego, co niosą elementy, skleja ostatnie słowo jednego bloku z pierwszym słowem następnego." }, + "Detail.format/html.formatting": { "hash": "sha256-6d6322985c6f", "other": "Jak ułożona jest strona. zminifikowany stawia cały plik w jednej linii i kończy go bez znaku nowej linii. z wcięciami stawia przed wierszem dwie spacje za każdy element, w którym wiersz siedzi, a tekst pre zostawia bez zmian. Wszystkie trzy wyglądają tak samo. Program czytający wierszami dostaje stronę zminifikowaną jako jeden wiersz - bufio.Scanner w Go odmawia wiersza dłuższego niż 64 KB, chyba że dostanie większy bufor - a tekst czytany przez łączenie tego, co niosą elementy, skleja ostatnie słowo jednego bloku z pierwszym słowem następnego. Wcięcia razem z zagnieżdżeniem dokładają przed każdym wierszem więcej spacji z każdym poziomem, więc zagnieżdżenie 1000 daje stronę co najmniej 1,9 MB." }, "Detail.format/html.inline": { "hash": "sha256-83c55b41b56a", "other": "Ile obrazków pokazuje strona. Każdy jest prawdziwym plikiem swojego formatu wpisanym w stronę jako adres data:, więc otwarcie strony niczego nie pobiera. Leżą nad blokami, każdy w osobnym akapicie, a każdy mówi w tekście alt, który to obrazek." }, "Detail.format/html.inline_format": { "hash": "sha256-0c0288233958", "other": "Format obrazków. Dowolny format, którego pliki są obrazkami, na przykład png, jpg albo gif. Chromium pokazuje każdy z nich poza tiff." }, "Detail.format/html.inline_size": { "hash": "sha256-2433610ca9da", "other": "Jak duży jest każdy obrazek. W stronie jest zapisany w base64 w jednej linii, mniej więcej o jedną trzecią większy. lxml, a z nim BeautifulSoup, pomija obrazek większy niż około 7,1 MB, chyba że kazano mu czytać ogromne drzewa. Na Linuksie lxml czytał taką stronę ponad 20 sekund, a przy obrazku 15 MB zgubił też resztę strony." }, + "Detail.format/html.links": { "hash": "sha256-c07164812721", "other": "Ile odnośników niesie strona, na liście nad obrazkami i blokami. Przechodzą po kolei przez 18 rodzajów i potem się powtarzają - odnośnik do samej listy, względny, do katalogu wyżej, od korzenia, samo zapytanie, bezwzględny z zapytaniem i fragmentem, http, bez schematu, z kodowaniem procentowym, domena międzynarodowa, port, mailto:, tel:, ftp:, pobranie, nowe okno, nofollow i data: - a tekst każdego mówi, którym jest. Każdy adres leży w domenie zarezerwowanej na przykłady. DOMPurify z ustawieniami domyślnymi wyjmuje adres data: i atrybut target, który otwiera nowe okno, a przy milionie odnośników zabrakło mu 8 GB pamięci, podczas gdy lxml przeczytał je w 3 sekundy. Katalog wszystkich elementów ma sześć własnych odnośników, których to ustawienie nie liczy." }, "Detail.format/html.nesting": { "hash": "sha256-cac70df12b4b", "other": "W ilu elementach div, jeden w drugim, siedzi treść. 0 umieszcza ją wprost w body, a etykieta zostaje na zewnątrz, więc strona dalej mówi, czym jest. Chromium zachowuje 511 poziomów i wyjmuje głębsze na zewnątrz, a Nu HTML Checker uznaje tak głęboką stronę za błąd. lxml gubi treść strony zagnieżdżonej na około 250 poziomów i zostawia tylko etykietę, a jsdom, parse5 i DOMPurify mogą wyczerpać stos przy zapisie strony od około 2 000 poziomów." }, "Detail.format/html.structure": { "hash": "sha256-40b00d44ba38", "other": "Czy plik jest całą stroną, czy tylko blokami, które stałyby w jej środku. Fragment nie ma doctype, elementu html ani body, czyli tyle, ile naprawdę trzyma pole treści albo treść e-maila. Jest dużo mniejszy, więc najmniejszy fragment jest wyraźnie mniejszy od najmniejszej strony." }, "Detail.format/html.styles": { "hash": "sha256-072a4d7b066a", "other": "Czy strona niesie CSS i gdzie. osadzone wstawia reguły do elementu style, w części head strony i na samym początku fragmentu. w atrybutach wstawia te same reguły w atrybut style każdego nagłówka, akapitu, listy, tabeli i cytatu, którymi strona jest wypełniona, a strona wygląda tak samo. DOMPurify z domyślnymi ustawieniami usuwa element style, nie wpisując go na listę usuniętych, i zostawia atrybuty. text_content z lxml i BeautifulSoup na html5lib liczą reguły jako tekst strony. Katalog wszystkich elementów dostaje reguły osadzone i żadnych atrybutów." }, @@ -428,7 +429,7 @@ "Label.bbox": { "hash": "sha256-708f44b1bd98", "other": "Ramka (bbox)" }, "Label.bit_depth": { "hash": "sha256-cc62f2e89a82", "other": "Głębia bitowa" }, "Label.body": { "hash": "sha256-6ccaa6415b5e", "other": "Treść" }, - "Label.bom": { "hash": "sha256-e39338101951", "other": "BOM" }, + "Label.bom": { "hash": "sha256-1471d8c7a9cf", "other": "BOM" }, "Label.bulk": { "hash": "sha256-3140100df131", "other": "Przesyłanie masowe" }, "Label.bytes": { "hash": "sha256-4d8000301fcc", "other": "Bajty" }, "Label.change_interval": { "hash": "sha256-e99a12a7d82d", "other": "Odstęp zmian obrazu" }, @@ -481,6 +482,7 @@ "Label.limit": { "hash": "sha256-674b0ed54bf7", "other": "Limit" }, "Label.line_ending": { "hash": "sha256-c4f7d0b83542", "other": "Koniec linii" }, "Label.line_endings": { "hash": "sha256-c948fba2b0dd", "other": "Końce linii" }, + "Label.links": { "hash": "sha256-9024c197d849", "other": "Odnośniki" }, "Label.message_depth": { "hash": "sha256-8a18b09b1f37", "other": "Głębokość wiadomości" }, "Label.methods": { "hash": "sha256-8696622f3441", "other": "Metody" }, "Label.modified": { "hash": "sha256-e8ce5dcaf408", "other": "Zmodyfikowano" }, diff --git a/internal/gui/text/locale/said/en.json b/internal/gui/text/locale/said/en.json index f3439f0e..4c554071 100644 --- a/internal/gui/text/locale/said/en.json +++ b/internal/gui/text/locale/said/en.json @@ -1786,9 +1786,14 @@ "other": "it has to be {{.Basic}} or {{.Every}}" }, "html.HasToBeFormatting": { - "description": "Said by internal/format/htmlfile - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Lines}}, {{.Minified}}, written the way the program writes them - keep each spelled exactly that way.", - "hash": "sha256-932932bf4b96", - "other": "it has to be {{.Lines}} or {{.Minified}}" + "description": "Said by internal/format/htmlfile - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Lines}}, {{.Minified}}, {{.Indented}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-4dc4e0b5ce30", + "other": "it has to be {{.Lines}}, {{.Minified}} or {{.Indented}}" + }, + "html.HasToBeLinks": { + "description": "Said by internal/format/htmlfile - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Max}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-ed9c43e64c22", + "other": "it has to be a whole number from 0 to {{.Max}}" }, "html.HasToBeNesting": { "description": "Said by internal/format/htmlfile - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Max}}, written the way the program writes them - keep each spelled exactly that way.", @@ -1821,9 +1826,9 @@ "other": "the file holds whole blocks and every picture in base64, which is about a third larger than the picture itself, and together they need that much" }, "html.MinimumWithSettings": { - "description": "Said by internal/format/htmlfile - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Nesting}}, {{.Characters}}, {{.Elements}}, {{.BOM}}, {{.Styles}}, written the way the program writes them - keep each spelled exactly that way.", - "hash": "sha256-fe52ba18f1bf", - "other": "the file holds whole blocks and what {{.Nesting}}, {{.Characters}}, {{.Elements}}, {{.BOM}} and {{.Styles}} add to them, and together they need that much" + "description": "Said by internal/format/htmlfile - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Nesting}}, {{.Characters}}, {{.Elements}}, {{.Links}}, {{.BOM}}, {{.Formatting}}, {{.Styles}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-8e1412a53466", + "other": "the file holds whole blocks and what {{.Nesting}}, {{.Characters}}, {{.Elements}}, {{.Links}}, {{.BOM}}, {{.Formatting}} and {{.Styles}} add to them, and together they need that much" }, "html.ThePictureCannotBe": { "description": "Said by internal/format/htmlfile - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Format}}, {{.Err}}, written the way the program writes them - keep each spelled exactly that way.", diff --git a/internal/gui/text/locale/said/pl.json b/internal/gui/text/locale/said/pl.json index b83ee7e3..dfd71d4c 100644 --- a/internal/gui/text/locale/said/pl.json +++ b/internal/gui/text/locale/said/pl.json @@ -355,14 +355,15 @@ "html.FragmentLeavesOut": { "hash": "sha256-47bec476fafe", "other": "Fragment nie ma głowy ani szkieletu strony, więc nie ma w nim elementów {{.Elements}}." }, "html.HasToBeCharacters": { "hash": "sha256-135f7b4d551c", "other": "musi to być {{.ASCII}} albo {{.International}}" }, "html.HasToBeElements": { "hash": "sha256-d471c09eb934", "other": "musi to być {{.Basic}} albo {{.Every}}" }, - "html.HasToBeFormatting": { "hash": "sha256-932932bf4b96", "other": "musi to być {{.Lines}} albo {{.Minified}}" }, + "html.HasToBeFormatting": { "hash": "sha256-4dc4e0b5ce30", "other": "musi to być {{.Lines}}, {{.Minified}} albo {{.Indented}}" }, + "html.HasToBeLinks": { "hash": "sha256-ed9c43e64c22", "other": "musi to być liczba całkowita od 0 do {{.Max}}" }, "html.HasToBeNesting": { "hash": "sha256-ed9c43e64c22", "other": "musi to być liczba całkowita od 0 do {{.Max}}" }, "html.HasToBeStructure": { "hash": "sha256-1a83745fa4b2", "other": "musi to być {{.Document}} albo {{.Fragment}}" }, "html.HasToBeStyles": { "hash": "sha256-41656b094378", "other": "musi to być {{.None}}, {{.Embedded}} albo {{.Inline}}" }, "html.MinimumFragment": { "hash": "sha256-e716d4f229d7", "other": "fragment zawiera całe bloki, a jeden z nich potrzebuje tyle" }, "html.MinimumPage": { "hash": "sha256-3745440d090a", "other": "strona zawiera nagłówek, treść i całe bloki, a po jednym z nich potrzeba tyle" }, "html.MinimumWithPictures": { "hash": "sha256-af57b5e328fe", "other": "plik zawiera całe bloki i każdy obrazek w base64, który jest mniej więcej o jedną trzecią większy od samego obrazka, a razem potrzebują tyle" }, - "html.MinimumWithSettings": { "hash": "sha256-fe52ba18f1bf", "other": "plik zawiera całe bloki i to, co dokładają do nich {{.Nesting}}, {{.Characters}}, {{.Elements}}, {{.BOM}} i {{.Styles}}, a razem potrzebują tyle" }, + "html.MinimumWithSettings": { "hash": "sha256-8e1412a53466", "other": "plik zawiera całe bloki i to, co dokładają do nich {{.Nesting}}, {{.Characters}}, {{.Elements}}, {{.Links}}, {{.BOM}}, {{.Formatting}} i {{.Styles}}, a razem potrzebują tyle" }, "html.ThePictureCannotBe": { "hash": "sha256-2c602605a691", "other": "html: obrazek {{.Format}} nie może powstać: {{.Err}}" }, "html.UseOneOfThePictureFormats": { "hash": "sha256-a6669f6abe6b", "other": "Użyj jednego z: {{.Formats}}." }, "json.HasToBeStyle": { "hash": "sha256-c1343585d922", "other": "musi to być {{.Indented}}, {{.Minified}} albo {{.RecordPerLine}}" }, diff --git a/internal/oracle/strict.py b/internal/oracle/strict.py index 1e689107..a4c44752 100644 --- a/internal/oracle/strict.py +++ b/internal/oracle/strict.py @@ -767,6 +767,7 @@ def html_settings(text, shape, settings, nl): fail(f"the html check was told elements={elements!r}, which is not one this tool writes") if "script" in names: fail("the page carries a script, which no setting of this format writes") + html_links(bare, settings, nl) html_pictures(bare, settings, nl) @@ -857,23 +858,119 @@ def html_picture_verdict(kind, n, raw): def html_layout(text, settings): """The layout formatting ordered, and what ends a line of it: a page of lines ends every one of them, the last included, and a page of one line - has no line break anywhere and no newline at the end.""" + has no line break anywhere and no newline at the end. An indented page is a + page of lines with two spaces in front of each for every element it sits + in - checked here line by line, and given back without them, so every check + after this one reads the page it would read written in lines.""" layout = settings.get("formatting", "lines") if layout == "minified": breaks = text.count("\n") + text.count("\r") if breaks: fail(f"formatting=minified was ordered and the page has {breaks} line breaks") - return "" - if layout != "lines": + return "", text + if layout not in ("lines", "indented"): fail(f"the html check was told formatting={layout!r}, which is not one this tool writes") # Asked here and not left to the ASCII check, which sees a carriage return # only on a page of ASCII - one in a page of other scripts went through, and # a fragment of CRLF lines ended in a traceback further down (review of #179). if "\r" in text: - fail(f"formatting=lines was ordered and the page has {text.count(chr(13))} carriage returns, where its lines end in LF") + fail(f"formatting={layout} was ordered and the page has {text.count(chr(13))} carriage returns, where its lines end in LF") if not text.endswith("\n") or text.count("\n") < 2: - fail("formatting=lines was ordered and the page is not lines ending in a line break") - return "\n" + fail(f"formatting={layout} was ordered and the page is not lines ending in a line break") + return "\n", html_indentation(text, layout == "indented") + + +HTML_VOID = {"area", "base", "br", "col", "embed", "hr", "img", "input", "link", "meta", "source", "track", "wbr"} + + +def html_indentation(text, indented): + """Every line indented by the elements open where it starts - two spaces a + level, one level fewer for a line that starts by closing one - or, on a + page of lines, by nothing. The text of pre is text and takes none. Given + back without the indentation. + + The depth is counted here from the tags themselves, written from the + standard's list of void elements rather than from the generator, so the two + cannot agree on a wrong one.""" + out, depth, in_pre = [], 0, False + tag = re.compile(r"<(/?)([a-zA-Z][a-zA-Z0-9-]*)([^>]*)>") + for number, line in enumerate(text[:-1].split("\n"), start=1): + if in_pre: + out.append(line) + else: + body = line.lstrip(" ") + lead = depth - 1 if body.startswith(" 1000000: + fail(f"the html check was told links={count!r}, which is not a whole number from 0 to 1000000") + want = int(count) + head = '" + nl, start) + if end < 0: + fail(f"links={want} was ordered and the list of links is never closed") + # The list is its links and nothing else. Read item by item, a list closed + # only at the end of the page carried every block inside it and still + # counted right, in the two layouts no indentation is checked in (review of + # #185). Matches that do not overlap and add up to the whole list are the + # whole list. + inside = text[start + len(head):end] + items, covered = [], 0 + for m in re.finditer(r'
  • ]*>[^<]+
    • ' + re.escape(nl), inside): + items.append(m.group(1)) + covered += m.end() - m.start() + if covered != len(inside): + fail(f"links={want} was ordered and the list of links holds something other than links") + if len(items) != want: + fail(f"links={want} was ordered and the list carries {len(items)} links") + for n, href in enumerate(items): + if href != HTML_LINKS[n % len(HTML_LINKS)]: + fail(f"link {n + 1} goes to {href!r}, and the kinds in turn make {HTML_LINKS[n % len(HTML_LINKS)]!r}") + host = HTML_LINK_HOST.match(href) + if host and not (host.group(1) in HTML_LINK_HOSTS or host.group(1).endswith(".example")): + fail(f"link {n + 1} goes to {host.group(1)}, which is not a domain kept for examples") # The elements the style rules are for, written here from the setting's @@ -984,9 +1081,9 @@ def check_html(data, settings): shape = settings.get("structure", "document") if shape not in ("document", "fragment"): fail(f"the html check was told structure={shape!r}, which is not one this tool writes") - nl = html_layout(text, settings) - html_styles(text, shape, settings) - html_settings(text, shape, settings, nl) + nl, flat = html_layout(text, settings) + html_styles(flat, shape, settings) + html_settings(flat, shape, settings, nl) opens = text.lower().startswith("") closes = text.rstrip().endswith("") diff --git a/web/public/ar/formats/index.html b/web/public/ar/formats/index.html index 91ddff25..cba4a969 100644 --- a/web/public/ar/formats/index.html +++ b/web/public/ar/formats/index.html @@ -671,6 +671,11 @@

    الإعدادات التي تقبلها كل صيغة

    elements basic, every + + + links + 0 - 1000000 + bom @@ -679,7 +684,7 @@

    الإعدادات التي تقبلها كل صيغة

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/cs/formaty/index.html b/web/public/cs/formaty/index.html index 651c7ce9..5729e5a0 100644 --- a/web/public/cs/formaty/index.html +++ b/web/public/cs/formaty/index.html @@ -672,6 +672,11 @@

    Nastavení, která každý formát přijímá

    elements basic, every + + + links + 0 - 1000000 + bom @@ -680,7 +685,7 @@

    Nastavení, která každý formát přijímá

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/de/formate/index.html b/web/public/de/formate/index.html index 5825ad81..577025a4 100644 --- a/web/public/de/formate/index.html +++ b/web/public/de/formate/index.html @@ -674,6 +674,11 @@

    Einstellungen, die jedes Format kennt

    elements basic, every + + + links + 0 - 1000000 + bom @@ -682,7 +687,7 @@

    Einstellungen, die jedes Format kennt

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/es/formatos/index.html b/web/public/es/formatos/index.html index 91faaad5..bb296e69 100644 --- a/web/public/es/formatos/index.html +++ b/web/public/es/formatos/index.html @@ -675,6 +675,11 @@

    Ajustes que admite cada formato

    elements basic, every + + + links + 0 - 1000000 + bom @@ -683,7 +688,7 @@

    Ajustes que admite cada formato

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/formats/index.html b/web/public/formats/index.html index 1db9b6ab..43c99266 100644 --- a/web/public/formats/index.html +++ b/web/public/formats/index.html @@ -673,6 +673,11 @@

    Settings each format accepts

    elements basic, every + + + links + 0 - 1000000 + bom @@ -681,7 +686,7 @@

    Settings each format accepts

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/fr/formats/index.html b/web/public/fr/formats/index.html index ffa77ce1..fcb88e86 100644 --- a/web/public/fr/formats/index.html +++ b/web/public/fr/formats/index.html @@ -675,6 +675,11 @@

    Réglages que chaque format accepte

    elements basic, every + + + links + 0 - 1000000 + bom @@ -683,7 +688,7 @@

    Réglages que chaque format accepte

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/hi/formats/index.html b/web/public/hi/formats/index.html index a77113b0..3c707385 100644 --- a/web/public/hi/formats/index.html +++ b/web/public/hi/formats/index.html @@ -673,6 +673,11 @@

    हर फ़ॉर्मैट की स्वीकार की जा elements basic, every + + + links + 0 - 1000000 + bom @@ -681,7 +686,7 @@

    हर फ़ॉर्मैट की स्वीकार की जा formatting - lines, minified + indented, lines, minified diff --git a/web/public/id/format/index.html b/web/public/id/format/index.html index 20d21738..8f71b495 100644 --- a/web/public/id/format/index.html +++ b/web/public/id/format/index.html @@ -673,6 +673,11 @@

    Pengaturan yang diterima setiap format

    elements basic, every + + + links + 0 - 1000000 + bom @@ -681,7 +686,7 @@

    Pengaturan yang diterima setiap format

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/it/formati/index.html b/web/public/it/formati/index.html index 55573f73..8405e9f2 100644 --- a/web/public/it/formati/index.html +++ b/web/public/it/formati/index.html @@ -675,6 +675,11 @@

    Impostazioni che ogni formato accetta

    elements basic, every + + + links + 0 - 1000000 + bom @@ -683,7 +688,7 @@

    Impostazioni che ogni formato accetta

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/ja/formats/index.html b/web/public/ja/formats/index.html index ec0c2bd7..961add9a 100644 --- a/web/public/ja/formats/index.html +++ b/web/public/ja/formats/index.html @@ -665,6 +665,11 @@

    形式ごとに受け付ける設定

    elements basic, every + + + links + 0 - 1000000 + bom @@ -673,7 +678,7 @@

    形式ごとに受け付ける設定

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/ko/formats/index.html b/web/public/ko/formats/index.html index ba69d828..98261b1c 100644 --- a/web/public/ko/formats/index.html +++ b/web/public/ko/formats/index.html @@ -666,6 +666,11 @@

    각 형식이 받는 설정

    elements basic, every + + + links + 0 - 1000000 + bom @@ -674,7 +679,7 @@

    각 형식이 받는 설정

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/nl/formaten/index.html b/web/public/nl/formaten/index.html index 862ac708..11c9992d 100644 --- a/web/public/nl/formaten/index.html +++ b/web/public/nl/formaten/index.html @@ -675,6 +675,11 @@

    Instellingen die elk formaat accepteert

    elements basic, every + + + links + 0 - 1000000 + bom @@ -683,7 +688,7 @@

    Instellingen die elk formaat accepteert

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/pl/formaty/index.html b/web/public/pl/formaty/index.html index f5d08d56..3b619c65 100644 --- a/web/public/pl/formaty/index.html +++ b/web/public/pl/formaty/index.html @@ -673,6 +673,11 @@

    Ustawienia, które przyjmuje każdy format

    elements basic, every + + + links + 0 - 1000000 + bom @@ -681,7 +686,7 @@

    Ustawienia, które przyjmuje każdy format

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/pt-br/formatos/index.html b/web/public/pt-br/formatos/index.html index 6fad45d4..6637d2c8 100644 --- a/web/public/pt-br/formatos/index.html +++ b/web/public/pt-br/formatos/index.html @@ -674,6 +674,11 @@

    Configurações que cada formato aceita

    elements basic, every + + + links + 0 - 1000000 + bom @@ -682,7 +687,7 @@

    Configurações que cada formato aceita

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/ro/formate/index.html b/web/public/ro/formate/index.html index dfd44c71..4a5c6211 100644 --- a/web/public/ro/formate/index.html +++ b/web/public/ro/formate/index.html @@ -674,6 +674,11 @@

    Setările pe care le acceptă fiecare format

    elements basic, every + + + links + 0 - 1000000 + bom @@ -682,7 +687,7 @@

    Setările pe care le acceptă fiecare format

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/ru/formats/index.html b/web/public/ru/formats/index.html index 581a1b2e..fcf6d020 100644 --- a/web/public/ru/formats/index.html +++ b/web/public/ru/formats/index.html @@ -673,6 +673,11 @@

    Настройки, которые принимает каждый форм elements basic, every + + + links + 0 - 1000000 + bom @@ -681,7 +686,7 @@

    Настройки, которые принимает каждый форм formatting - lines, minified + indented, lines, minified diff --git a/web/public/th/formats/index.html b/web/public/th/formats/index.html index 70a91baf..69c059ad 100644 --- a/web/public/th/formats/index.html +++ b/web/public/th/formats/index.html @@ -671,6 +671,11 @@

    การตั้งค่าที่แต่ละรูปแบบร elements basic, every + + + links + 0 - 1000000 + bom @@ -679,7 +684,7 @@

    การตั้งค่าที่แต่ละรูปแบบร formatting - lines, minified + indented, lines, minified diff --git a/web/public/tr/bicimler/index.html b/web/public/tr/bicimler/index.html index 65c3609a..1c08fdad 100644 --- a/web/public/tr/bicimler/index.html +++ b/web/public/tr/bicimler/index.html @@ -673,6 +673,11 @@

    Her biçimin kabul ettiği ayarlar

    elements basic, every + + + links + 0 - 1000000 + bom @@ -681,7 +686,7 @@

    Her biçimin kabul ettiği ayarlar

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/uk/formats/index.html b/web/public/uk/formats/index.html index b355a8aa..a31cf03b 100644 --- a/web/public/uk/formats/index.html +++ b/web/public/uk/formats/index.html @@ -673,6 +673,11 @@

    Налаштування, які приймає кожен формат

    elements basic, every + + + links + 0 - 1000000 + bom @@ -681,7 +686,7 @@

    Налаштування, які приймає кожен формат

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/vi/dinh-dang/index.html b/web/public/vi/dinh-dang/index.html index 2879ac8b..66c7979e 100644 --- a/web/public/vi/dinh-dang/index.html +++ b/web/public/vi/dinh-dang/index.html @@ -671,6 +671,11 @@

    Các thiết lập mỗi định dạng nhận

    elements basic, every + + + links + 0 - 1000000 + bom @@ -679,7 +684,7 @@

    Các thiết lập mỗi định dạng nhận

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/zh-hans/formats/index.html b/web/public/zh-hans/formats/index.html index 952a0956..c336447c 100644 --- a/web/public/zh-hans/formats/index.html +++ b/web/public/zh-hans/formats/index.html @@ -665,6 +665,11 @@

    每种格式接受的设置

    elements basic, every + + + links + 0 - 1000000 + bom @@ -673,7 +678,7 @@

    每种格式接受的设置

    formatting - lines, minified + indented, lines, minified diff --git a/web/public/zh-hant/formats/index.html b/web/public/zh-hant/formats/index.html index f39e4907..e9cf30f8 100644 --- a/web/public/zh-hant/formats/index.html +++ b/web/public/zh-hant/formats/index.html @@ -665,6 +665,11 @@

    每種格式接受的設定

    elements basic, every + + + links + 0 - 1000000 + bom @@ -673,7 +678,7 @@

    每種格式接受的設定

    formatting - lines, minified + indented, lines, minified