Linux Open Source Presentation
Okash Khawaja
okash.khawaja at gmail.com
Tue Aug 13 03:50:16 EDT 2019
Hi,
Thank you very much and sorry about late reply. I emailed OSS
organisers and the date for submissions of slides is 14 August.
I have updated the slides based on your feedback. I have attached both
PDF and text only version.
Best regards,
Okash
On Wed, Jul 24, 2019 at 8:06 AM Samuel Thibault
<samuel.thibault at ens-lyon.org> wrote:
>
> Hello,
>
> Okash Khawaja, le dim. 21 juil. 2019 18:04:31 +0100, a ecrit:
> > slide 1: What is speakup?
> > -------------------------
> > - piece of software that reads out what's on console and what you type with keyboard
>
> It will probably be interesting to mention the two modes: it speaks text
> as it gets printed by applications, but the user can also use speakup
> shortcuts to read what was printed by the application and is still
> visible on the screen.
>
> > slide 3: Present
> > -----------------
> > TODO: tty based vs directly accessing ports
>
> Yes, that's an interesting point.
>
> Before the tty support, the port knocking was obviously posing
> problems of conflicting with the existing serial drivers, and supporting
> only legacy serial ports.
>
> You can mention that speakup is particular in that it's a purely
> software driver that needs to operate the serial port. Thus a line
> discipline, thus what you implemented.
>
> > TODO: console content and keyboard snooping done properly
>
> You can indeed mention the proper use of console text fetching, and
> keyboard notification registration.
>
> > TODO: what distros ship with speakup?
>
> I must admit that I don't know :)
>
>
> You can also mention that proper unicode support was done, so that all
> languages can now be supported seamlessly.
>
> > slide 3: Future
> > ----------------
> >
> > - move from staging to mainline
> > - fix bugs
> > - new features: usb drivers, auto load
> > - TODO: what else for future? ask mailing list
>
> I think there is at least still work to be done for internationalization
> of speakup messages and such things.
>
> > slide 5: Core:
> > --------------
> > - main.c contains speakup core
> > - main.c + supporting files yield speakup.ko
> > - upon loading:
> > - perform some initialisations (spk_vars, add virtual keyboard, initialise an internal representation of every console etc)
> > - initialise sysfs interface
> > - register tty ldisc
> > - synth_init() (TODO: what is this?)
>
> It tries to initialize the synths given on the command line.
>
> > - speakup kthread (mainly for synth->catch_up()
>
> You can explain that speakup uses buffering: keyboard and console
> notifiers queue stuff to be spoken, and the speakup kthread
> progressively pushes that to a serial port or the software synthesis.
>
> > slide 6: Core - keyboard and vt:
> > --------------------------------
> > [TODO]
>
> I'm not sure whether you want to detail much of that. You could make
> some rehearsal to see whether you will have the time to detail here.
>
> Perhaps that's the place for just explaining the buffering I mentioned
> above.
>
> > slide 8: struct spk_syth:
> > -------------------------
> > TODO: reproduce its code here and go through members one by one
>
> It would be tedious to cover it all. You can probably just cover it in a
> few related groups like: probe/release,
> read_buff_add/synth_immediate/catch_up/flush, is_alive, get_index
>
> > slide 9: struct spk_io_ops:
> > ---------------------------
> > - struct spk_io_ops
>
> That one you can probably cover it all.
>
> > slide 12: Software Synths: speakup_dummy:
> > -----------------------------------------
> > - useful for development purposes
>
> Yes, that's an important point to make: you don't need an actual synth
> to do development.
>
> > - typical development set up with speakup_dummy:
> > - qemu with serial port tied to a file on host
> > - load speakup_dummy with dev param pointing to the serial port
>
> It'd be useful that you provide the ready-to-run commands so people can
> try it out.
>
> Samuel
-------------- next part --------------
slide 0:
-------
Linux Speakup Makes Linux Talk to Users: Past and Future
Okash Khawaja / Contributor to Speakup
@binarydebt
slide 1: What we will cover
---------------------------
- What is speakup
- Past, present and future
- Some technical details:
- overall arch
- hardware synths
- ttyio architecture
- Development process
- Q&A
slide 2: What is speakup?
-------------------------
- Piece of software that reads out what's on console and what
you type with keyboard
- Two modes:
- Read text printed by applications
- Whole set of keyboard shortcuts to control the output,
including going back and reading what was
previously printed by application and still on screen
- Linux only
- Entirely kernel-based, although exposes interface to user
space
- *Starts at early boot and lasts until shutdown*
slide 3: History
-----------------
- Originally written by a blind person
- Started by Kirk Reiser in 1998
- Merged into Linux staging directory in late 2010
- Now work is underway to move it out of staging into kernel proper
- External device or ISA card for additional serial port
slide 4: Present
----------------
- tty based vs directly accessing ports
- unique design because it's purely a software driver
and needs to operate on serial port
- console content and keyboard snooping done properly
- register_vt_notifier()
- register_keyboard_notifier()
- proper unicode support added: all languages can be
supported seamlessly
- ships with following distros: Debian, [TODO: which
others?]
slide 5: Future
----------------
- Move from staging to mainline
- Fix bugs: garbled text
- New features: usb drivers, auto load,
- internationalisation of speakup messages
- More here: https://github.com/bytefire/speakup/issu
es
slide 6: Overall Architecture
-----------------------------
- core (speakup.ko)
- synths (speakup_*.ko)
- flow of data: {console, keyboard} -> notifiers -> circular buffer -> kthread -> synths (serial port or software interface)
slide 7: Core
-------------
- main.c contains speakup core
- main.c + supporting files yield speakup.ko
- upon loading:
- perform some initialisations (spk_vars, add virtual keyboard, initialise
an internal representation of every console etc)
- initialise sysfs interface
- register tty ldisc
- synth_init()
- register keyboard notifier
- register vt notifier
- speakup kthread (take from buffer and push to serial or software
interface)
slide 8: Device Architecture
----------------------------
- Two types of synths: hardware and software
- Inside hardware, two types of devices: internal ISA cards (old) and external
- We will focus on external hardware synths and software synths
slide 9: Device Architecture - Hardware Synths
----------------------------------------------
- Different synth chipsets: DoubleTalk, Apollo, DecTalk, Braille 'n Speak and
more
- Connect using serial protocol: either serial port RS-232 or serial over USB
- Pure USB support planned
slide 10: Device Architecture: Support from core
-------------------------------------------------
struct spk_synth {
...
struct spk_io_ops *io_ops;
int (*probe)(struct spk_synth *synth);
void (*release)(void);
const char *(*synth_immediate)(struct spk_synth *synth,
const char *buff);
void (*catch_up)(struct spk_synth *synth);
void (*flush)(struct spk_synth *synth);
int (*is_alive)(struct spk_synth *synth);
void (*read_buff_add)(u_char);
unsigned char (*get_index)(struct spk_synth *synth);
...
};
slide 11: Device Architecture: CPU to Synth
--------------------------------------------
- TTY layer to send data to device
struct spk_io_ops {
int (*synth_out)(struct spk_synth *synth, const char ch);
int (*synth_out_unicode)(struct spk_synth *synth, u16 ch);
void (*send_xchar)(char ch);
void (*tiocmset)(unsigned int set, unsigned int clear);
unsigned char (*synth_in)(void);
unsigned char (*synth_in_nowait)(void);
void (*flush_buffer)(void);
};
slide 12: Device Architecture: Synth to CPU
--------------------------------------------
- Why synth-to-CPU traffic?
- Use line discipline to communicate from device to CPU
- include/linux/tty_ldisc.h:struct tty_ldisc_ops
- Methods we care about are:
- open()
- close()
- receive_buf2()
slide 13: Speaking from early boot
-----------------------------------
- How it worked before: inb(), outb() etc
- For TTY, handle to tty_struct is needed
- Usually obtained from user space by opening /dev/ttyUSB0
- But we want to start speaking before there
is file system or even user space
slide 14: Speaking from early boot
-----------------------------------
- Need to open tty from kernel space
- Introduced tty_kopen() and tty_kclose()
- Needed addressing potential synchronisation issues with user-space
open of same tty
- Affected user-kernel interface because an
open() of a tty could now return -EBUSY
slide 15: Device Architectur: Putting it together
----------------------------------------------------
- struct spk_synth
- struct spk_io_ops
- tty + ldisc
[TODO: diagram]
slide 16: Software Synths
-------------------------
Two software synths for different
motivations:
- speakup_soft.ko
- speakup_dummy.ko
slide 17: Software Synths: speakup_soft.ko
-------------------------------------------
- Like any other synth but doesn't push speech to device
- Instead exposes an interface to user space
- Misc device /dev/softsynth that can be read by a user
space program in the same way as hardware synth
would read and then speak.
- speakup.ko <-> speakup_soft.ko <->/dev/softsynth <->
espeakup <-> espeak -> physical speakers
- Limitations: requires user space to be up before it can
start speaking and doesn't speak all shutdown
messages
slide 18: Software Synths: speakup_dummy.ko
-------------------------------------------
- Useful for development purposes
- Typical development set up with speakup_dummy:
- qemu with serial port tied to a file on host
- load speakup_dummy with dev param pointing to the serial port
slide 19: Software Synths: speakup_dummy.ko
-------------------------------------------
- Get all output in text
- E.g. useful to debug garbled speech
- No extra layers (e.g. user space
components) involved
slide 20: Development Process
-----------------------------
- Like general kernel development process but simpler and friendlier :)
- Internal reviews! thanks to Samuel
- Helpful community:
- Testing
- Help with getting hardware devices
- Tech savvy
slide 21: Development Process (contd.)
--------------------------------------
- qemu: compile on host or inside qemu VM
- speakup_dummy for most cases hardware synth for hardware features / debugging
- careful to load correct driver for the synth
- speakup_apollo.ko on double talk
- "just goes to show you when you give double
talk to a god" - one of the mailing list replies
slide 22: Development Process (contd.)
--------------------------------------
- great way to learn
- real community impact
- we're working towards moving it out of staging into main kernel
- plenty of interesting work ahead:
- Complete transition out of staging
- Support for USB synths
- USB autoload
- Bug fixes possible races
- Internationalisation of speakup messages
- See https://github.com/bytefire/speakup/issues for more
slide 23: Q&A
--------------
Q&A
More information about the Speakup
mailing list