MIL OSI Translation. Region: Russian Federation –
Source: Novosibirsk State University – Novosibirsk State University –
The 20th anniversary Total Dictation will be written by artificial intelligence for the first time. New open system “Scribe”, which was developed by a researcher at the Laboratory of Applied Digital Technologies International Scientific and Educational Mathematical Center of NSU and co-founder of the Siberian Neural Networks startup, Ivan Bondarenko, will take part in this annual event on an equal basis with human intelligence, and just like other participants in the Total Dictation, he will be given a grade depending on the number of grammatical and punctuation errors made.
— The idea to create the “Scribe” system came to me a year ago. Initially, it was intended to help applicants for a scientific degree in the difficult work of compiling a transcript of a dissertation council meeting. According to my plan, the new system had to be proficient in the Russian language and correctly record what the announcer reads, correctly placing punctuation marks, observing the rules of grammar and spelling of the Russian language. Usually, this work takes a person quite a lot of time, since the duration of the audio recording of the dissertation council meeting is an hour or two. Artificial intelligence would save the dissertation candidate from this work, significantly saving time. So Scribe was originally conceived as a shorthand system. Of course, as a speech recognizer it is not ideal. On the one hand, the “Scribe” does not always correctly recognize words, the percentage of errors is 20-30%, but even in such a situation, the presence of an interlinear text significantly facilitates the dissertation candidate’s work on compiling the text of the transcript – he does not need to compile it from scratch, he just needs to make corrections. On the other hand, “The Scribe” can be criticized for being too verbatim in its recording, so stylistic editing will be required,” said Ivan Bondarenko.
Ivan Bondarenko worked on the open system “Scribe” in his free time. The scientist was helped by a graduate student Faculty of Mechanics and Mathematics Daniil Grebenkin, who created an augmentator, with the help of which it became possible to artificially expand the training set for the “Scribe”, adding noise models to the signals obtained in pure conditions, and recording speech under conditions similar to real ones. Also, two fourth-year students have recently joined the project – Roman Derunets and Mikhail Klementyev, who are helping in the preparation of a new version of “The Scribe”. “The Scribe” has already passed a number of tests, now he faces a serious test – to complete a task intended for human intelligence. These kinds of tests fit into the general paradigm of testing artificial intelligence.
— Back in 2017, an article was published by Gary F. Marcus, a professor in the Department of Psychology at New York University and the founder of Geometric Intelligence, in which he wondered how best to distinguish artificial intelligence from human intelligence, given that the Turing test no longer allows for this effective due to its extreme simplicity and known cases of "missing the target". One of the new ways to test artificial intelligence described in this article is to use the same standardized tests in science and other subjects that schoolchildren and college students take on machines. Dictation is a standard test of literacy and language proficiency. And “Total Dictation,” precisely because of its totality, makes it possible to collect a large amount of statistical material on human errors and compare these errors with the errors of speech “artificial intelligence,” explained Ivan Bondarenko.
The Scribe was prepared for the Total Dictation in advance. A week ago, its creator and student assistants began working on updates in order to put the updated version of the system up for testing on April 20. The updates are aimed at improving the Scribe's knowledge of punctuation and text structure. He was “taught” to place punctuation marks more correctly, to break the text into sentences and paragraphs, and also to separate the dictation text itself from the dictator’s remarks and comments. Ivan Bondarenko noted that “The Scribe” has become smarter, but nevertheless, Total Dictation is unlikely to receive a high rating. According to his forecasts, the artificial intelligence will write the words themselves correctly, but it is unlikely to cope with the placement of punctuation marks – punctuation still remains its weak point. Updates can reduce the number of such errors, but this may not be enough for a positive assessment – to get a “D”, it is enough to make 5 errors. However, scientists are not at all interested in the Scribe’s assessments or the competition between artificial intelligence and human intelligence, but in the analysis and comparison of the mistakes they make.
Note; This information is raw content directly from the source of the information. This is exactly what the source states and does not reflect the position of MIL-OSI or its clients.
Please note; This information is raw content directly from the information source. It is accurate to what the source is stating and does not reflect the position of MIL-OSI or its clients.
EDITOR'S NOTE: This article is a translation. Apologies should the grammar and or sentence structure not be perfect.