November 24, 2017 at 10:11Development for Sailfish OS: working with sound using the example of the DayTimer application
- Tutorial
Hello! This article is the continuation of a series of articles on application development for the Sailfish OS mobile platform. This time we will focus on developing an application for counting days to an event (for example, to graduation, New Year or Birthday), selected by the user. For each event, the user can add a description and record an audio note. We start the article with a description of the user interface, and then we analyze the operation of the application.
The main application screen is a list of user events. Each element of the list contains the name and date of the event, clicking on it opens a screen with information about the event. List items also have a context menu that allows you to delete the event.
The main screen of the application also contains a pull-out menu with one item that opens a dialog for adding a new note.
The dialog for adding / editing events contains text fields for entering the name and description, buttons for setting the date and time of the event, and components for recording and playing audio notes, which we will examine in detail in this article.
The number of remaining days is displayed on the "cover" of the application, and when the date and time of the event comes, the user receives a notification.
Note in advance that in order for the emulator to be able to record and play sound files, including through QML, it is necessary that the qt5-qtmultimedia-plugin-mediaservice-gstmediacapture and qt5-qtmultimedia-plugin-mediaservice- packages are installed on it gstmediaplayer .
The easiest way to do this is to add them to the Requires section of the .yaml file so that they are installed with the application:
In addition, in the latest versions of the emulator, the volume level is set to the minimum by default, therefore, in order to hear the sound files played, you must run the following commands on the emulator:
You can execute them by connecting to the emulator via ssh:
The functionality of recording and listening to a voice note was placed in a separate AudioML layer component built into the page .
Work with audio occurs from the pages of viewing and editing events. On the editing page, the component is displayed with a record button, which has two states: start and stop recording (standard and red record button , respectively), a record playback button ( play ), which changes to a stop button during listening, and a download bar.
To start recording, the icon-m-dot icon was selected , and for the convenience of the user, a custom button was created to stop recording ( red-dot ). To start and stop listening, the icon-m-media and icon-m-tabs buttons were selected, respectively. Available icons can be viewed on the official Jolla website or in the documentation that comes with the SDK.
Recording starts when the button is pressed and ends when the button is pressed again or after the set time has elapsed (default 120 seconds), after recording the button for listening becomes available, and the note can be rewritten an unlimited number of times.
On the viewing page in the AudioPlayer component, recording is not available, the operation of all other elements does not change.
To store our voice notes, use the " DayTimer " folder located in the directory " / home / nemo /", which is a system in Sailfish OS. However, users can listen to and delete their audio notes outside the application. The application checks the availability of previously created files and, if they are missing, blocks the function of listening to a voice note for this event.
To record a voice note, the QAudioRecorder class is used , and to listen to the QML type MediaPlayer from the Qt Multimedia library . To work with the library you need to register in the pro-file:
It is worth noting that when developing for Sailfish OS, all the audio settings functionality provided by the QAudioEncoderSettings class is also available , which contains many useful methods, for example, the bitRate () method that returns the bitrate value of an audio file. If desired, we can fix the existing bitrate to the one we need, using the method of setting the bitrate by the value of setBitRate () .
Learn to use all available codecs can use the method supportedAudioCodecs () class QAudioEncoderSettingsControl , which also features a method codecDescription () , which gives a description of each codec separately. You can change the audio codec using setCodec ()same class.
In addition, in the class you can find methods for setting quality, encoding, and much more.
We will analyze how the work with the sound file database takes place. When you open the event adding page, a temporary file is created in which the created records are saved. Further actions depend on the decision of the user, since the page is a dialog box. The file is deleted if the user has canceled the changes, or his name is recorded (or overwritten - when overwriting the voice note) in the application database, if the user has confirmed the changes.
Since there is no standard QML component that writes sound to a file, we created and registered our own QML component VoiceRecorder using the methods of the QAudioRecorder class , as described in a previous article. In addition to the methods described in detail below, the class contains the audioInput property , which stores the actual name of the audio file (you can change it using the setAudioInput () method ). Properties and some methods of the QMediaRecorder class are also used .
Next, we move on to the analysis of the methods of the QAudioRecorder class that we use . This class contains an audioRecorder object , with the help of which the record () and stop () methods are responsible for recording a voice note. Also VoiceRecorder defined property isRecording, which determines whether the recording is currently ongoing or not. The recordingChanged signal notifies of a change in the value of this property.
The class has methods for writing ( record () ) and stop ( stop () ), which use states from the QMediaRecorder class .
To overwrite a voice memo or delete an event in a class, methods for checking for the presence of audio ( isExistVoice () ) and its removal ( removeVoice () ) using the standard methods of the class QFile ( exists () and remove () , respectively) are used.
In the createVoiceName () method, using the mkpath () method of the QDir class , a folder is created in which voice notes will be stored. The QUuid class is used to create audio file names , so the names will be unique.
We’ll connect the AudioPlayer component on the viewing and editing pages and move on to parsing its QML components, for which the VoiceRecorder class was created .
All work on recording a voice note is tied directly to the recordButton button , the code of which is given below. It is worth noting the visible property : it is determined based on the page on which we connect the AudioPlayer , and sets the availability of the button for recording using flagRecordButtonVisible .
In the AudioPlayer component , we created an object of type MediaPlayer , and also set the isRecord property , which is used as a flag to set the maximum value of the ProgressBar . If the flag is true , then the maximum value of the ProgressBar is 120 seconds, otherwise the duration of the audio file. A detailed description of the operation of the ProgressBar will be given below.
The following is the button code for listening to a voice note. The player.source () method in the setSource () function sets the path to the file to be played.
During implementation, we were faced with the fact that the first time you listened to a dubbed voice note, the old recording was played, since the standard player.stop () method did not unload the old file from the buffer. To solve this problem, we used the seek () method , which sets the position from which listening will begin.
To visualize recording and playback of a voice note, a ProgressBar is used , in which the time is counted using the Timer component . The maximum value of the ProgressBar is set depending on whether the voice memo is recorded or listened (using the component property isRecord) In the first case, this is the 120 seconds mentioned earlier, and in the second, the length of the voice note you are listening to. It is worth noting that the value itself is specified in milliseconds divided by the timer interval. When the maximum value of the ProgressBar is reached , recording or listening ends.
As a result, an application was created that allows you to create events with voice notes and track the remaining number of days before them. The source code for the DayTimer application is published on BitBucket, and the application itself is available for download to OpenRepos for everyone.
Technical issues can also be discussed on the channel of the Russian-speaking community Sailfish OS in Telegram or the VKontakte group .
Authors: Svetlana Yurkina, Marina Kuchma, Irina Moskovkina
Application user interface
The main application screen is a list of user events. Each element of the list contains the name and date of the event, clicking on it opens a screen with information about the event. List items also have a context menu that allows you to delete the event.
The main screen of the application also contains a pull-out menu with one item that opens a dialog for adding a new note.
The dialog for adding / editing events contains text fields for entering the name and description, buttons for setting the date and time of the event, and components for recording and playing audio notes, which we will examine in detail in this article.
The number of remaining days is displayed on the "cover" of the application, and when the date and time of the event comes, the user receives a notification.
 |  |  |
Work with audio
Note in advance that in order for the emulator to be able to record and play sound files, including through QML, it is necessary that the qt5-qtmultimedia-plugin-mediaservice-gstmediacapture and qt5-qtmultimedia-plugin-mediaservice- packages are installed on it gstmediaplayer .
The easiest way to do this is to add them to the Requires section of the .yaml file so that they are installed with the application:
Requires:
# Following packages are needed to playback and record audio on the Sailfish OS Emulator.
- qt5-qtmultimedia-plugin-mediaservice-gstmediaplayer
- qt5-qtmultimedia-plugin-mediaservice-gstmediacaptureIn addition, in the latest versions of the emulator, the volume level is set to the minimum by default, therefore, in order to hear the sound files played, you must run the following commands on the emulator:
pactl set-sink-mute sink.emulator 0
pactl set-sink-volume sink.emulator 40000You can execute them by connecting to the emulator via ssh:
ssh nemo@localhost -p 2223 -i <Путь к SailfishOS SDK>/vmshare/ssh/private_keys/SailfishOS_Emulator/nemoThe functionality of recording and listening to a voice note was placed in a separate AudioML layer component built into the page .
Description of the operation of the AudioPlayer component
Work with audio occurs from the pages of viewing and editing events. On the editing page, the component is displayed with a record button, which has two states: start and stop recording (standard and red record button , respectively), a record playback button ( play ), which changes to a stop button during listening, and a download bar.
To start recording, the icon-m-dot icon was selected , and for the convenience of the user, a custom button was created to stop recording ( red-dot ). To start and stop listening, the icon-m-media and icon-m-tabs buttons were selected, respectively. Available icons can be viewed on the official Jolla website or in the documentation that comes with the SDK.
Recording starts when the button is pressed and ends when the button is pressed again or after the set time has elapsed (default 120 seconds), after recording the button for listening becomes available, and the note can be rewritten an unlimited number of times.
On the viewing page in the AudioPlayer component, recording is not available, the operation of all other elements does not change.
To store our voice notes, use the " DayTimer " folder located in the directory " / home / nemo /", which is a system in Sailfish OS. However, users can listen to and delete their audio notes outside the application. The application checks the availability of previously created files and, if they are missing, blocks the function of listening to a voice note for this event.
Additional project settings
To record a voice note, the QAudioRecorder class is used , and to listen to the QML type MediaPlayer from the Qt Multimedia library . To work with the library you need to register in the pro-file:
QT += multimedia. It will also be useful to indicate the dependency in the yaml-file (section Requires ): qt5-qtmultimedia .Additional options when working with audio
It is worth noting that when developing for Sailfish OS, all the audio settings functionality provided by the QAudioEncoderSettings class is also available , which contains many useful methods, for example, the bitRate () method that returns the bitrate value of an audio file. If desired, we can fix the existing bitrate to the one we need, using the method of setting the bitrate by the value of setBitRate () .
Learn to use all available codecs can use the method supportedAudioCodecs () class QAudioEncoderSettingsControl , which also features a method codecDescription () , which gives a description of each codec separately. You can change the audio codec using setCodec ()same class.
In addition, in the class you can find methods for setting quality, encoding, and much more.
Record audio to file
We will analyze how the work with the sound file database takes place. When you open the event adding page, a temporary file is created in which the created records are saved. Further actions depend on the decision of the user, since the page is a dialog box. The file is deleted if the user has canceled the changes, or his name is recorded (or overwritten - when overwriting the voice note) in the application database, if the user has confirmed the changes.
Since there is no standard QML component that writes sound to a file, we created and registered our own QML component VoiceRecorder using the methods of the QAudioRecorder class , as described in a previous article. In addition to the methods described in detail below, the class contains the audioInput property , which stores the actual name of the audio file (you can change it using the setAudioInput () method ). Properties and some methods of the QMediaRecorder class are also used .
Next, we move on to the analysis of the methods of the QAudioRecorder class that we use . This class contains an audioRecorder object , with the help of which the record () and stop () methods are responsible for recording a voice note. Also VoiceRecorder defined property isRecording, which determines whether the recording is currently ongoing or not. The recordingChanged signal notifies of a change in the value of this property.
class VoiceRecorder : public QObject {
Q_OBJECT
Q_PROPERTY(bool isRecording READ isRecording NOTIFY recordingChanged)
public:
explicit VoiceRecorder(QObject *parent = nullptr);
bool isRecording() const;
// ...
signals:
void recordingChanged();
private:
QAudioRecorder *m_audioRecorder;
bool m_recording;
};
The class has methods for writing ( record () ) and stop ( stop () ), which use states from the QMediaRecorder class .
void VoiceRecorder::record(const QString &name) {
audioRecorder->setOutputLocation(QUrl(name));
if (audioRecorder->state() == QMediaRecorder::StoppedState) {
audioRecorder->record();
b_recording = true;
emit recordingChanged();
}
}
void VoiceRecorder::stop () {
if (audioRecorder->state() == QMediaRecorder::RecordingState) {
audioRecorder->stop();
b_recording = false;
emit recordingChanged();
}
}
To overwrite a voice memo or delete an event in a class, methods for checking for the presence of audio ( isExistVoice () ) and its removal ( removeVoice () ) using the standard methods of the class QFile ( exists () and remove () , respectively) are used.
In the createVoiceName () method, using the mkpath () method of the QDir class , a folder is created in which voice notes will be stored. The QUuid class is used to create audio file names , so the names will be unique.
QString VoiceRecorder::createVoiceName() {
QDir path = QDir::home();
const QString storagePath = QStringLiteral("%1/DayTimer").arg(QDir::homePath());
if (!path.exists(storagePath)) {
path.mkpath(storagePath);
}
QUuid u = QUuid::createUuid();
QString str = QStringLiteral("%1/DTVoice%2.wav").arg(storagePath).arg(u.toString());
return str;
}
Working with audio from a QML page
We’ll connect the AudioPlayer component on the viewing and editing pages and move on to parsing its QML components, for which the VoiceRecorder class was created .
AudioPlayer {
id: voice
property bool flagRecordButtonVisible: true
}
All work on recording a voice note is tied directly to the recordButton button , the code of which is given below. It is worth noting the visible property : it is determined based on the page on which we connect the AudioPlayer , and sets the availability of the button for recording using flagRecordButtonVisible .
IconButton {
id: recordButton
icon.source: "image://theme/icon-m-dot"
visible: flagRecordButtonVisible
onClicked: {
progressBar.value = 0;
isRecord = true;
}
// ...
}
In the AudioPlayer component , we created an object of type MediaPlayer , and also set the isRecord property , which is used as a flag to set the maximum value of the ProgressBar . If the flag is true , then the maximum value of the ProgressBar is 120 seconds, otherwise the duration of the audio file. A detailed description of the operation of the ProgressBar will be given below.
Item {
id: audioPlayer
property bool isRecord: false
MediaPlayer {
id: player
}
}
The following is the button code for listening to a voice note. The player.source () method in the setSource () function sets the path to the file to be played.
IconButton {
id: playButton
// ...
onClicked: {
module.setSource();
isRecord = false;
timer.stop();
progressBar.value = 0;
player.seek(player.duration);
recordButton.enabled = true;
playButton.icon.source = "image://theme/icon-m-media";
// ...
}
}
// ...
function setSource() {
if (voiceRecorder.isExistVoice(tempName)) {
player.source = tempName;
} else {
player.source = voiceName;
}
}
During implementation, we were faced with the fact that the first time you listened to a dubbed voice note, the old recording was played, since the standard player.stop () method did not unload the old file from the buffer. To solve this problem, we used the seek () method , which sets the position from which listening will begin.
To visualize recording and playback of a voice note, a ProgressBar is used , in which the time is counted using the Timer component . The maximum value of the ProgressBar is set depending on whether the voice memo is recorded or listened (using the component property isRecord) In the first case, this is the 120 seconds mentioned earlier, and in the second, the length of the voice note you are listening to. It is worth noting that the value itself is specified in milliseconds divided by the timer interval. When the maximum value of the ProgressBar is reached , recording or listening ends.
ProgressBar {
id: progressBar
// ...
maximumValue: {
if (player.duration > 0 && !isRecord) {
return player.duration / 100;
} else { return 1200 }
}
Timer {
id: timer
interval: 100
repeat: true
onTriggered: {
if (progressBar.value >= progressBar.maximumValue) {
// ...
timer.stop();
progressBar.value = 0;
// ...
} else {
progressBar.value += 1
}
}
}
Conclusion
As a result, an application was created that allows you to create events with voice notes and track the remaining number of days before them. The source code for the DayTimer application is published on BitBucket, and the application itself is available for download to OpenRepos for everyone.
Technical issues can also be discussed on the channel of the Russian-speaking community Sailfish OS in Telegram or the VKontakte group .
Authors: Svetlana Yurkina, Marina Kuchma, Irina Moskovkina
