[PATCH] tty: resolve contention over tty device between user and kernel space

[Okash Khawaja]

Hi,

I have attached the patch to address the issue when opening a device
from user space when it is already opened by kernel. I have used
tty->flags field as that seems to fit the purpose.

However, there are two things left unaddressed:

1. when kernel opens the device while it is opened from user space it
will still create a new tty_struct through a call to tty_init_dev.
2. how will kernel "release" the device? e.g. decrement tty->count and
undo other things it did in tty_open_by_driver

As I understand it exporting tty_open_by_driver (and this patch) are
fine when the device is opened at boot time and held until shutdown,
i.e. when speakup is built into kernel and not as a module. When built
as a module, it could open the tty device while it is user opened and it
may stop using it without ever decrementing tty->count, preventing user
space from opening it.

Thanks,
Okash

---
drivers/staging/speakup/spk_ttyio.c | 2 ++
drivers/tty/tty_io.c | 7 +++++++
include/linux/tty.h | 1 +
3 files changed, 10 insertions(+)

--- a/drivers/staging/speakup/spk_ttyio.c
+++ b/drivers/staging/speakup/spk_ttyio.c
@@ -151,6 +151,7 @@ static int spk_ttyio_initialise_ldisc(in
pr_warn("speakup: Failed to set hardware flow control\n");
}

+ set_bit(&tty->flags, TTY_KOPENED);
tty_unlock(tty);

ret = tty_set_ldisc(tty, N_SPEAKUP);
@@ -258,6 +259,7 @@ void spk_ttyio_release(void)
speakup_tty->ops->close(speakup_tty, NULL);

tty_ldisc_flush(speakup_tty);
+ clear_bit(&speakup_tty->flags, TTY_KOPENED);
tty_unlock(speakup_tty);
tty_ldisc_release(speakup_tty);
}
--- a/drivers/tty/tty_io.c
+++ b/drivers/tty/tty_io.c
@@ -2137,6 +2137,13 @@ retry_open:

tty_add_file(tty, filp);

+ if (test_bit(TTY_KOPENED, &tty->flags)) {
+ tty_warn(tty, "device already opened by kernel\n");
+ tty_unlock(tty);
+ tty_release(inode, filp);
+ return -EBUSY;
+ }
+
check_tty_count(tty, __func__);
tty_debug_hangup(tty, "opening (count=%d)\n", tty->count);

--- a/include/linux/tty.h
+++ b/include/linux/tty.h
@@ -363,6 +363,7 @@ struct tty_file_private {
#define TTY_NO_WRITE_SPLIT 17 /* Preserve write boundaries to driver */
#define TTY_HUPPED 18 /* Post driver->hangup() */
#define TTY_LDISC_HALTED 22 /* Line discipline is halted */
+#define TTY_KOPENED 23 /* Already opened by kernel */

/* Values for tty->flow_change */
#define TTY_THROTTLE_SAFE 1