Python serial gui part 1

[Jeff_T]

I have a bit of fun here with a serial gui that also works as a useful tool. The explanation is minimalistic but the code is split over five pieces to aid in understanding how it's put together.
I think you can gain more by adding piece by piece to the app but I have also attached the complete code at the end. The code is versatile and can be modified to however you want, such as modified for binary data or to create a .csv data collection application
In part 1 we will lay out the template for our application.
Part 1
The first six lines are our imports, we can add additional imports/modules later as needed, these six modules are the basis for this example.
The tkinter (TK interface) package is installed with the standard Python installation and this is what we will use to build the GUI, there are also have two imports for threading and one for our serial routines.

Code Select Expand

from tkinter import *
from tkinter import scrolledtext , ttk
from tkinter import messagebox
import threading
from threading import Thread
import serial.tools.list_ports
The next two lines are the tkinter gui program loop, every line of code we write over the next few parts will go between these two lines.

Code Select Expand

window = Tk()
window.mainloop()
If you run the program now you should see the beginning of our application.

[Jeff_T]

Part 2
This part is a small adjustment of the window size and title, initialization of the serial object and a couple of python lists to contain baud rates and port numbers. The following lines are inserted at the top and inside the tkinter program loop

Code Select Expand

window.title("Serial")
window.geometry("700x450")
event = threading.Event()
ser = serial.Serial()
baud_list = ["9600", "19200", "38400", "57600", "115200"]
port_list = []
ports = list(serial.tools.list_ports.comports())
for p in ports:
    port_list.append(p.name)
Immediately after this we will "pencil in" nine functions leaving them empty for now.

Code Select Expand

def open_port(port_number):
    pass
def close_port():
    pass
def thread_start():
    pass
def thread_break():
    pass
def rx_thread():
    pass
def send_data():
    pass
def clear():
    pass
def close_app():
    pass
def on_closing():
    pass


[Jeff_T]

Part 3
Here we add the widgets, there are six in total and the syntax is similar for all of them. We start by selecting a widget and giving it a name, the widget has parameters which start with the parent and follow with various options.
Most importantly the widgets will not be visible until you tell your program the location you want them placed. There are a couple of ways to place the widgets but when you choose a method it is best to stick with it. My example is using the "grid" method.
So these are the six widgets, a text box, a label frame (to contain the user controls), two combo boxes and two buttons.
Place the widget code inside the the tkinter loop and below the nine functions.

Code Select Expand

txt_out = scrolledtext.ScrolledText(window,width=50,bg='black',fg='white')
txt_out.grid(row=0, column=1, rowspan=2, pady=10)

comms_frame = LabelFrame(window, text="Port Select",bg='gray')
comms_frame.grid(row=0, column=0, padx=10)

ports_cmb = ttk.Combobox(comms_frame, values=port_list, width=8)
ports_cmb.grid(row=0, column=1, pady=5, padx=10)
if len(port_list)>0:
    ports_cmb.current(0)
else:
    ports_cmb.set("No Port")

baud_cmb = ttk.Combobox(comms_frame, values=baud_list, width=8)
baud_cmb.grid(row=1, column=1, pady=5, padx=10)
baud_cmb.current(0)

btn1 = Button(comms_frame, text="OpenPort",bg='light gray',command=lambda: open_port(ports_cmb.get()), width=10)
btn1.grid(row=0, column=0, pady=5, padx=5)

btn2 = Button(comms_frame, text="ClosePort",command = lambda:thread_break(), width=10)
btn2.grid(row=1, column=0, pady=5, padx=5)
Enter this code and run the application to see the results

[Jeff_T]

Part 4
Now we can start filling in a few functions starting with port_open(), this function is called by the btn1 widget which also passes  the selected COM port as an argument.
Apart from initializing the serial port COM number and baud rate this function starts a thread which continuously monitors the serial input buffer, the function called to start the thread is thread_start().
To exit the rx thread when we close the app there is a threading event which is initially made clear (event.clear()), later when we close the application we set the event (event.set()). If we close the port while still trying to read the rx buffer we will get an exception so stopping the thread before we close the port is a must. The threading event  is an important  piece of the open and close process.

Code Select Expand

def port_open(port_number):
    event.clear()
    if not ser.isOpen():
        try:
            ser.port = port_number
            ser.baudrate = baud_cmb.get()
            ser.open()
            thread_start()
        except serial.SerialException as e:
            txt_out.insert(END,e)
If our port opens successfully thread_start() is called and the thread is started.

Code Select Expand

def thread_start():
    t1 = Thread(target=rx_thread, args=(), daemon=True)
    t1.start()
The rx_thread() is the heart of the application, it's function for this example is to read an ascii string that is terminated with a newline character. By changing the code in this thread we can read many different protocols but for now we will stick to the original theme of a newline terminated ascii string.

Code Select Expand

def rx_thread():
    txt_out.insert(END, "Session open......." + "\n")
    while True:
        if event.is_set():
            break
        if ser.inWaiting():
            try:
                inp = ser.readline().decode('utf-8')
                txt_out.insert(END,inp)                       
            except UnicodeDecodeError as e:
                pass
            except serial.SerialException as e:
                pass
So next lets deal with stopping the rx thread and closing the serial port, this is done by the btn2 widget which calls the function thread_break(). This function simply sets the threading event (event.set()) to stop the rx thread and follows by calling the close_port() function.

Code Select Expand

def thread_break():
    event.set()
    close_port()
Finally close_port() which closes the serial port and displays a notification.

Code Select Expand

def port_close():
    if ser.isOpen():
        ser.close()
        txt_out.insert(END, "\n" + "Session closed.\n" )
What we have so far will open and close a port and will display received data in the app's text box

[Jeff_T]

Part 5
There is so much more that can be done with this template but just for now I want to clean up three of the four remaining functions. The function clear() deletes all text from the text box and for this we need an additional button widget so add this to the list of button widgets with its grid location.

Code Select Expand

btn3 = Button(master=comms_frame, text="Clear",command = lambda:clear(), width=10)
btn3.grid(row=2, column=0, pady=5, padx=5)
now the three functions

Code Select Expand

def clear():
    txt_out.delete("1.0", END)
   
def close_app():
    window.destroy()
   
def on_closing():
    if messagebox.askokcancel("Quit", "Are you sure you want to exit?"):
        event.set()
        close_app()
Finally the following line will determine what happens when the application closes.

Code Select Expand

window.protocol("WM_DELETE_WINDOW", on_closing)
Try it out with a microcontroller,  here is an Arduino test sketch @115200 baudrate.

Code Select Expand

void setup() {
  Serial.begin(115200);
  Serial.println();
}

void loop() {
    Serial.println("Hello world!");
    delay(1000);
    }
Part 6

Here is perhaps the simplest way to write data to a .csv file, it will probably be adequate for many mcu applications. To collect the data we need two additional lines in the rx_thread() function, these two lines check to see if the string has the correct header which I have designated as the dollar sign '$', if that checks out that data is passed to the function that formats the data and appends the data to a '.csv' file.

Here is the modified rx_thread()

Code Select Expand

def rx_thread():
    global idx
    txt_out.insert(END, "Session open......." + "\n")
    ser.timeout=200
    ser.read()
    while True:
        if event.is_set():
            break
        if ser.inWaiting():
            print(ser.inWaiting(),'    ',idx)
            idx+=1
            try:
                inp = ser.readline().decode('utf-8')
                txt_out.insert(END,inp)                       
            except UnicodeDecodeError as e:
                pass
            except serial.SerialException as e:
                pass
           
            if inp[0]=="$":
                add_data(inp)
and the extra function to write the data to a file.

Code Select Expand

def add_data(inp):
    data_string=inp.replace("$,","",1)
    try:
      with open('datalogger.csv', "a" ,newline="") as file:
          file.write(data_string)
    except IOError as e:
            pass
Here is an example of formatting the data to log using an Arduino.

Code Select Expand

void setup() {
  // put your setup code here, to run once:
Serial.begin(9600);

}

void loop() {
  // put your main code here, to run repeatedly:
Serial.println("$,1,2,3,4");
delay(1000);
}

[Chris Savage]

Jeff,

Interesting application...reminds me of the QuickStart Communicator application I did some years back. But I do have one important question...what development tool did you use for this? Visual C#?

[Jeff_T]

Hi Chris, I used the Thonny Python IDE, I have another very similar application that collects information over wifi and I am considering posting that also.

The intention on both of these apps is to collect data and write the data to a csv file in other words a datalogger.

I have the additional code it is basically just one extra function, I will add it to post#4 a little later. The serial gui I use a lot for testing Arduino input and output, I use it  more than I do the serial monitor in the Arduino IDE.

[Chris Savage]

I hope you don't mind, I linked it in your reply so others can download it. 

[Chris Savage]

I also replied to your DM a few days ago. See the General Forum for a tutorial.

[Jeff_T]

Thanks Chris, I saw the DM and tutorial but at the time I just couldn't get it lol. I'll try again later I don't think it's a biggie  right now. In my mind something like this, if it's new  to you, then you're better off copying and pasting each part  and trying to fully understand what each part does.

Just expanding on the topic a little do you ever watch YouTube electronic tutorials, there is a guy does a lot with the Arduino and his name is Andreas Spiess aka The Guy with a Swiss accent, I came across one of his videos the other day from 9 years ago and what he did was take input from a microcontroller to act like a USB keyboard, he took this input to type values into the cells of an Excel worksheet which got me thinking about doing something similar.

MS Office 365 has it's own built in data streamer but if you own an older version of Office, like I still use a version of 2007 professional, your options are limited. So my solution was to use a VBA button on an Excel worksheet to call and run a Python script in the background that typed the logged data onto a worksheet. The Python app interfaces with a microcontroller using code almost identical to the code at the top of this thread. I am going to play around with this and see what it looks like replacing the serial connection with wifi. Do you think something like that would have value if I posted it here on the forum?

One last thing, for anyone that likes the ease of using Python there is a program that takes your Python sketches and converts them to a standalone executable (.exe), the program is called auto-py-to-exe and can be installed from the command line.

 https://pypi.org/project/auto-py-to-exe/

[Chris Savage]

Thanks Chris, I saw the DM and tutorial but at the time I just couldn't get it lol. I'll try again later I don't think it's a biggie  right now.

Would it help if I did an actual video tutorial of how to use the file attachment facility? I used to do stuff like that, but I guess I take it for granted that forums are so common now that everyone has used everything. I realize SMF is not as popular as it once was. Still, doing a video would only take a few minutes and I have the resources and time for that.

Just expanding on the topic a little do you ever watch YouTube electronic tutorials, there is a guy does a lot with the Arduino and his name is Andreas Spiess aka The Guy with a Swiss accent, I came across one of his videos the other day from 9 years ago and what he did was take input from a microcontroller to act like a USB keyboard, he took this input to type values into the cells of an Excel worksheet which got me thinking about doing something similar.

I do have several YouTube channels bookmarked and others show up in my feed, including many from the UK, like Big Clive, Great Scott and Julian Ilett. Sometimes I watch reviews of new tech and sometimes I get ideas from other people's projects to build my own.

As is typical with YouTube, for everything I watch, something similar or related will often be recommended, and as such I sometimes find new channels.

MS Office 365 has it's own built in data streamer but if you own an older version of Office, like I still use a version of 2007 professional, your options are limited. So my solution was to use a VBA button on an Excel worksheet to call and run a Python script in the background that typed the logged data onto a worksheet. The Python app interfaces with a microcontroller using code almost identical to the code at the top of this thread. I am going to play around with this and see what it looks like replacing the serial connection with wifi. Do you think something like that would have value if I posted it here on the forum?

Sure! Even when I had my old website with 1,500 members, there were some people who would message me referring to themselves as "lurkers" and not really interested in interacting with others. These days I am inclined to only provide answers to questions online where others can benefit from the information, though sometimes these tutorials or projects are inspired by something someone asked me. Check out the section, "Schoolhouse Greenhouse" from my Door Control – Part 1 tutorial. This was inspired by a tech support call I got at Parallax.

One last thing, for anyone that likes the ease of using Python there is a program that takes your Python sketches and converts them to a standalone executable (.exe), the program is called auto-py-to-exe and can be installed from the command line. https://pypi.org/project/auto-py-to-exe/

Very cool! I like that I create create stand-alone apps in Visual Studio, though I haven't done much in a while.