feature proposal for handwriting font realism

[Nick Levinson]

Many advertisements have headlines in script fonts in which a given character is identical
throughout the document. Real handwriting, however, has slight variations between
"e" and "e" or between "n" and "n", for example.
Try it yourself, especially by writing at your normal speed, not slower or more
meticulously than usually.

Less frequent uses include mimicking stone-cut fonts.

But this realism of variation disappears in computer graphics. One could change text to
several similar fonts, but most of us don't install several scripts that are
graphically coordinated as if they're from the same hand, if such fonts even exist.
Character-by-character substitution would be laborious anyway. Bitmap art programs are a
kludge at most, largely impracticable.

The solution I propose is automatic font-swapping within a subset of fonts.

A user would designate several fonts as being mutually swappable by the running
executable. The principal functionality could be performable mostly by the operating
system at an application's call commanded by the user.

Text, either selected or during input, would be manually assigned one set of fonts, the
same way a user normally selects a single font. Each character in the selection or input
stream would then be automatically assigned a font from that mutually-swappable set, a
font swapset.

Size and style would be unaffected, although size could be reconsidered for an
enhancement, because an approximately ten-percent size variation is likely to be more
realistic than pure consistency.

The sequence of auto-assignment of fonts and the first-character assignment would default
to random, with saving of the document preserving assignments as with traditional fonts.
Alternatively, the user could specify a sequence. In a hypothetical list of three fonts,
specifying
1,2,3,2,2
would cause the second font to be auto-assigned to 60 percent of the characters.

All characters would be affected in a simpler implementation, but better would be to
assign to whitespace and control characters whichever font the prior nonwhitespace
graphical character had, with an arbitrary assignment to the first character if it's
whitespace or nongraphical.

Since the same or similar glyphs are what human viewers are most sensitive to on this
issue, it would be better to override the sequence when the same or a similar glyph is
next encountered. That could be done by applying a clone of the sequence. Thus, for the
sequence
1,2,3
and the string
she said he said
the assignments (ignoring whitespace [in between]) would be
123 2123 12 3231
not
123 1231 23 1231
The former is better because each "s" would be unique and so would each
"h", each "e", each "a", each "i", and each
"d".

Glyph similarity with nonidenticality would be specified by the user with a default
specification offered for each alphabet or glyph set. A specification could be in a text
file line with spaces as separators (e.g., in part, "LEF Il1 bh ij mn bp vw O0 RPp Ss
cou GCOU"), as user-editable.

Most of the programming should be in one place in or near the OS, with apps slightly
rewritten to take the advantage.

Kernel growth should be minimized. Arguably, usually only graphic designers will care.
Therefore, a runtime-loadable user-loadable module or daemon should contain most of the
intelligence for handling this solution. Permanent OS hooks should be written to impose
the least overhead when the functionality is not needed during a boot or login session.

End-use applications should have little programming to support this facility, having just
enough to take advantage of it, just as end-use apps have little programming in support of
various kinds of printer features, but can use many kinds. Users may come to expect this
facility in several major applications on one machine, so it would be more efficient to
place the intelligence in one piece of software callable by all font-using applications or
callable by the OS at any font-using application's behest. Application designers may
have to expand the user interface a little. An app's UI offering a choice of fonts
should also offer a choice of swappable font sets, perhaps in a font menu submenu
that's dynamic.

Near where fonts are stored on disk, a directory should be dedicated to storing text files
designating swappable font sets. Multiple text files could support multiple font swapsets,
useful as different graphic designers have different artistic judgments about which fonts
should be in a given set, which nonidentical glyphs are similar, and how much size
variation should be tolerated.

The file should be a text file. From a hash to an end-of-line would be a comment. Lines
would be counted, each line implicitly numbered except for comment-only lines. The
following variables [(without equals signs)] should be supported, with one variable per
line:
version=
menuitem=
menuitemaltcmd=
menuitemctlcmd=
sequence=
font=

On either side of the equals sign, inline whitespace should be ignored.

The values of the above vars would be as follows:
* "version" would be the version of support for this file, to support future
expansion, with backwards compatibility supported. Edits to the file would not require
editing the version value. If a user wants versioning of the user's edits to the file,
a comment will do.
* "menuitem" could be read by an application and potentially assigned to naming
a menu item or any comparable functionality. Ordinarily, the filename would be the same,
but it needn't be, especially useful if a user wants to include in the menu item a
character that can't be in a filename, such as a slash, or include in the filename a
character that can't be in the menu item, such as an ampersand. This risks having two
menu items that are erroneously identical, but an application optionally could present an
error message to the user when starting the app and trying to populate a dynamic menu, and
the user could then edit the menuitem value in one file to uniqueness or both files would
be ignored for the menu.
* "menuitemaltcmd" could be read by an application and potentially assigned to
the first matching string in the menuitem value, so that a user could access the menu item
by keyboarding it with the alt key, as is typical for many menus. The default value would
be none and a failure to match would be treated like finding no match.
* "menuitemctlcmd" could be read by an application and potentially assigned to
the first matching string in the menuitem value, so that a user could access the menu item
by keyboarding it with the control key, as is typical for many menus. The default value
would be none and a failure to match would be treated like finding no match.
* "sequence" would default to random unless a value of random or a space-, tab-,
or comma-separated one-line list of numbers is specified. The numbers, which would not
need zero-padding, would correspond to font variables in order of appearance in the text
file.
* "font" would name one font, optionally with a full or partial path. If no path
is supplied, the default would be the path recognized for fonts by the OS. Multiple font
variables would normally be present. If no font was installed for a given font variable,
that variable would be ignored. If no font was installed for any font variable, the font
set would appear in the font menu only as unavailable (thus dimmed).

After fonts were thus assigned, the text file would no longer be needed for the
document's fonts.

Any fonts would be technologically compatible, but, to get full value, this would
encourage developing sets of closely similar fonts that plausibly could represent one
hand. Not many sets would be needed for designers to apply the system to graphic design.
For backward compatibility, each of the new (latter) fonts would work as single fonts do
now.

This facility may require that printers have more memory, because a page having many fonts
can cause some printers to choke. But this general swapfont feature would be effective
even with only two to four fonts representing handwriting, and many printers handle that
already.

If the number of fonts (each of which must be fully loaded into a printer's memory) is
a concern, I propose the multiglyph font (or another name if this is already taken). In
it, each character would be available in several glyphs. Those appearing more often in
running natural language (e.g., in English, the "e", or the letters in the
phrase "a sin to err", which appear more than others) could be rendered with
more glyphs in a single multiglyph font. Lower-case letters would need more glyphs than do
capitals, although capitals would need more than one glyph per character for all-capital
headlines. Such a multiglyph font should work like ordinary fonts for backwards
compatibility, by drawing only one glyph per character [(and always the same glyph)]
except where used as a multiglyph font. Thus, the multiglyph font-aware OS or program
would recognize an ordinary font as really a multiglyph font and unleash the
multfont's power. Various multiglyph fonts
 could be designed, but only one need be installed for graphic designers to start using
the technology.

To give graphic designers even more specific control, each glyph for a character would
have a unique identifier, so the designer could pick the glyph desired. The default
operation, however, would be for the designer to pick a multiglyph font and let the OS or
program assign glyphs just as it would from a font swapset.

The above was the first or original post to the like-named thread in a FedoraForum.org
forum (originally Rawhide until moved to Wibble). The post was by me, was posted there
April 9, 2012, and is copied here unedited except as bracketed and except that what I
originally called the multifont I now call the multiglyph font.

The 2d post (first reply) was about my choice of forum and is not relevant here (similar
discussions from other posts are also omitted, _infra_).

The 3d post was by sonoran, was posted the same day, and is quoted here in full:

"That daemon would make the Maytag repair man look like a hyperactive third grader.

"The glyph variation itself is a neat idea. But realistically it belongs in a scribus
or libreoffice plugin. You could accomplish the same thing using unicode and special
purpose script fonts with multiple glyphs per letter, and not have to alter apps or the os
at all."

The 4th post was by Gareth Jones, was posted April 10, 2012, and is quoted here in full:

"I only skim-read the original post, but isn't that what the alternate-glyphs
feature of modern fonts is intended for? As far as I know, the Linux font-rendering
libraries (FreeType, Pango etc.) are already aware of alternate glyphs, and use them when
needed systematically (e.g. character-sets where a glyph's rendering depends on its
context), but need explicit glyph instructions from higher-level software for more
artistic uses, as sonoran suggests. The developers of such programs are who to contact if
you want to request that feature.

"Actually, I remember reading something that suggested that XeTeX might already
support such things."

The 5th post was by me, was posted April 12, 2012, and is quoted here:

"I've seen titling capitals and maybe a few other same-character choices in a
font, but not for all characters in a font. Even if all-character-choice fonts exist,
those would still require manually assigning the character in the font to one character in
the string or input at a time, and that's very laborious. If the fonts I'm
proposing already exist, they're not being used in obvious contexts, probably because
of the labor. Computers are good at automation, and advantage should be taken of the
technology.

"Putting it into LibreOffice or some such would make the feature unavailable to users
of other applications, such as Gimp and graphics apps, and users use many apps to get
their work done. It should be available wherever text is typed and even moderately
manipulable for visual effect. A plugin for a particular app is probably not portable to
most apps on a user's machine, thus limiting. Putting it in one app would also make
font development less likely (there was a time when fonts were in apps but that design
model was abandoned). Putting the feature support where any font-using app can gain the
advantage is what will get it used the most.

"Unicode wouldn't work, since there's only one Unicode character for the
lower-case 'a', for example. Indeed, you'd want to rely on there being only
one Unicode designation to allow for the font design variations to be assigned to the
repeated single character.

"XeTeX does not seem to have that feature, according to the official website cited in
the Wikipedia article, but maybe I missed something.

". . . . I'm not concerned with the feature being in a specific [future] version
[of Fedora] . . . ."

The 6th post was by bob, was posted the same day, and is quoted here:

"Nick, . . . if [the thread is] . . . multi-OS (as you seem to indicate) . . . .
."

The 7th and 8th posts were by Dan, were apparently identical, and were posted the same day
and the seventh is quoted here:

"Afternoon, Nick.

". . . . [T]his is . . . an idea which is probably best expressed over on the fedora
project communication page. (developer's mailing list)

"Here it won't get a whole lot of attention by the folks who can actually do
something about it. . . . ."

The 9th post was by sonoran, was posted the same day, and is quoted here, including
quotations from my post/s:

"Originally Posted by Nick Levinson . . .

"'Even if all-character-choice fonts exist, those would still require manually
assigning the character in the font to one character in the string or input at a time, and
that's very laborious.'

"It would be done by the plugin and the logic would be straightforward. You could for
example base it on the most-recently-used form of the glyph, so that you would not have
identical glyphs appearing close together.

"Or you could base it on the adjoining or preceding glyphs - which would probably
most closely resemble actual handwriting. In other words, letter x assumes one form when
it follows letters a,f,t,h, and another form when it follows s,d, ... etc.

"Then for added realism you could combine the two heuristics, so that letter x
following letter a would not always be exactly the same.

"Quote:

"'Putting it into LibreOffice or some such would make the feature unavailable to
users of other applications'

"Not sure I understand what you mean. It is true that the plugin would be necessary
to compose the text, but if they had the font, then any app that can display unicode text
could properly display the script output the same way they display any other text.

"Quote:

"'Unicode wouldn't work, since there's only one Unicode character for the
lower-case "a", for example.'

"You would have multiple glyphs for lower-case 'a', as many variations as
necessary to achieve the desired realism. The plugin - having decided which variation of
'a' to use for best visual results - would output that variation's code in the
text. When the display software encountered that character code it would simply draw that
glyph the same way it draws any other character.

"The reason I'm arguing for this type of implementation is that it simply
isn't realistic to expect changes to the kernel and new text-file directories in every
os installation, just to achieve more realistic script display. You should take the
existing text display system and work from there - especially since what you want to
accomplish is achievable that way. Unicode is very powerful."

The 10th post was by DBelton, was posted April 13, 2012, and is quoted here (suspension
points & capitalization so in original & I think the original had a smiley not
shown here):

"Or a novel approach....

"IF you wish to have a realistic handwriting font you could... <*GASP*> pick up
a pencil and paper and write it."

The 11th post was by me, was posted April 14, 2012, and is quoted here:

"Which OS: I wasn't specific but it could start in the Fedora distro or any
distro. I happen to like and use Fedora (10 now and likely 15 soon) so I proposed it here.
Its functionality could be inspiring to any other distro or OS; Macintosh comes to mind.

"I didn't get the Maytag/hyperactivity comment. If the concern is that my
proposed feature would be too sluggish, it's only a design concept, and speeding it up
would be fine. As it is, it shouldn't noticeably slow Fedora down, since it should run
only when commanded by a user and most users would not use it for most documents or for
most parts of a document. Since font menus are dynamic already and populate quickly even
from large font lists, this need not add any significant delay to an application at that
stage. It'll be slower to use when the user weighs aesthetics and makes visual
judgments, but so is using Gimp instead of physical paintbrushes and scissors; and quality
work is often worth the care.

"Plugins work and I don't doubt the ability to design one, but I question how
useful a one-app plugin is for users of other apps. But if I misunderstand and one plugin
(not just a ports collection) could be available to any app, then a plugin is fine.

"Choosing a glyph according to its neighbors is a good idea that will probably
require more research into writing styles and is likely culturally specific. If either
approach leaves room for the other, that's good.

"While these new or regular fonts would be available to any application, the system
of choosing which of alternative glyphs in a font would apply to a particular Unicode code
point should be available to users of other apps. Thus, that logic needs to be where any
app can access it, in addition to the fonts themselves being available to where any
font-using app can access them.

"In response to 'it simply isn't realistic to expect changes to the kernel
and new text-file directories in every os installation, just to achieve more realistic
script display. You should take the existing text display system and work from there -
especially since what you want to accomplish is achievable that way. Unicode is very
powerful.': Not every OS installation, just in new ones with the feature; I wasn't
thinking of backporting and this should not interfere with normal use for users who
don't seek the feature. Nor replacing directories; backward compatibility with
existing fonts is good. Nor choosing a favorite glyph for 'x' but rather varying
it without having to vary by manually commanding a new variant with every letter, which is
what a user would have to do now (and which they don't do). Logic should be added to
allow automation of variation. Unicode is a set of characters that are each understood by
the public as being distinct in more than
 visual senses, usually as having different meanings or belonging to different contexts,
so expanding Unicode to encompass half a dozen ways of handwriting each character would
require reconceptualizing Unicode itself and maybe enlarging the number of bytes needed to
express each Unicode character and I don't think either is necessary. If one font
offers three glyphs for 't' and another font offers five, the logic could take
advantage of either without inserting missing-character or missing-glyph symbols or
ignoring available glyphs and without expanding or redefining Unicode.

"I think we agree on a lot and that we're about half way to agreement on the
concept.

"I categorize the 'pencil' idea with general principles of Luddites. Great
idea that applies to nearly everything done on computers. As to advertising headlines, so
many people have unappealing handwriting that the likelihood is that the noncomputer
solution would be to hire a painter for about the price of a Microsoft software package,
and the painter wouldn't be reusable for the fee."

The above is quoted from the forum as it stood on April 14 and the following is likewise
but as of April 21.

The 12th post was by Dan, was posted April 14, 2012, and is quoted here:

"Good deal, Nick. Give 'em the devil on their own doorstep, and something really
nice may come out of it.

"I look forward to seeing the concept in execution."

The 13th and, so far, last post was by sonoran, was posted April 16, 2012, and is quoted
here:

"Quote:

"Originally Posted by Nick Levinson . . .

"'I didn't get the Maytag/hyperactivity comment.'

"It was an old ad campaign featuring Jesse White - the idea was that Maytag
appliances were so good the repairman seldom had anything to do.

"Quote:

"'the system of choosing which of alternative glyphs in a font would apply to a
particular Unicode code point should be available to users of other apps. Thus, that logic
needs to be where any app can access it'

"My thought was that the logic to determine which glyph to use only needs to be
available when the text is first laid out, at which time the appropriate character code is
selected. When the time comes for any app to display that text it wouldn't need
anything but the font. So the plugin would only be required by apps like word processors
and page layout programs, and would not be necessary just to display the text.

"But it's your idea - you should run with it however you think best.

"(another variation occurred to me - the location of a glyph in a word. Most
handwriting fonts contain connectors at the bottom so that letters appear continuous when
displayed. This always looks wrong for a lowercase letter at the start of a word, people
just don't write that way.

"So the heuristic would check whether a glyph was lowercase at the start of a word
and use the appropriate variant if so. Something similar could be done for the last letter
in a word because people don't write glyphs with trailing connectors there,
either.)"

My reply to the last post (a new reply, not at the forum):

I agree, with one exception, that the logic is not necessary for text display (and
therefore it's not necessary for printing). If several fonts of today's font types
are available, then all that's needed is the assigning of a font to a character, and
that will exclusively determine a single glyph for the character. The exception is if
special fonts are designed using a new technology supported by this feature, a multiglyph
font, in which the "e" would have several glyphs in the one font, then, once a
glyph has been assigned, the only logic needed is that determining which glyph belongs to
a particular character at a particular position in a string, and even then the rest of the
logic, controlling the assignment process, would not be needed once the glyph has been
assigned. Further, even for the last case, the document could store the glyph selector
with the font identification that is with the character, in which case no logic is needed
outside of the
 multiglyph font and the document.

Positional adaptation is also a good idea; thank you, sonoran. In my observations,
connected-script fonts always put the point of connection at a constant height above the
base line and almost no one ordinarily writes that way.

[Corey Richardson]

On Sat, 21 Apr 2012 12:08:53 -0700 (PDT), Nick Levinson <nick_levinson(a)yahoo.com&gt;
wrote:
>  [snip lots of things] 
To me, as an amatuer graphic designer, this seems like a really easy way to
get pretty good-looking handwritten designs. As a programmer, though, this
seems like hell. You're saying that all of the multiple glyphs for a single
character would have the same metrics but slightly different style and a few
variations? Would this primarily be for on-screen or print purposes? I see it
being more useful for print since screen's DPI sucks majorly. For purely print
and with identical metrics, it wouldn't be too bad.

How would you have this implemented? Part of fontconfig or freetype?