Exercise Posture Tracking Using Computer Vision

So this is what I worked on last presentation. So if you were here here last time, you saw this already. Using object detection to track the barbell, a very straightforward computer vision problem. But with that, then you get these displacement curves and then you get velocity, purely just from this. So you can already go really far with just, object detection for Barbell. Again, very straightforward. You get the rep count, average rep, fastest, slowest. Okay. So I posted that on LinkedIn, right? It went viral. It got 2 50 ks impressions. Yeah. Yeah. And 4 4000 likes. I probably gained like 1000 followers. So the moral of the story is demonstrating Tinkers, and post it on LinkedIn. That's the moral of this story. Okay? Okay. So you saw that. It's just, you know, red box tracking really straightforward. But this time we're more ambitious, okay? We're going, we're trying to do something more crazy. This is relatively straightforward of computer vision. This 1 I'm really happy about because this this demo, I haven't really seen people do something quite like this. So before that, 2 off the shelf technologies that you can use free to use, I did not train them like off the shelf, you can just use them. So 1 from Google and 1 from RoboFlow. So, this type of pose landmark IDENTIFICATION is really common in these types of fitness tracking applications. So, they're designed they're designed to be very quick inference on your phone or whatever. Very lightweight, and it'll get you something like this. So you can imagine why I wanna use this information to, you know, filter out some things or have some frame of reference for the the deadlift. Right? And then I'm also using a brand new model from RoboFlow, open source, a very, very permissive license, like, you know, you can use it in your production code. Sorry, get their video to load, not my maybe not. But you saw it. It's like, just a state of the art segmentation model, using it off the shelf just to predict the person class, right? So, it took some experimentation to find the right model, but this is a good 1. I can I can vouch for it? Okay. So remember we had, these ingredients, right? So we had the bounding box for the, the barbell, this white line Sorry, can't really like zoom in super well, but we're Now I'm only looking at from the shoulder to the hip to the knee. So kind of creating, like, this greater than sign. That's this white line. Then I segment the person again, just calling the model to segment the person, and then this is what we can create. So if you can see it, getting that red line from the edge of the back because us as humans, we see in the video, it's like, oh, you know, the back edge is very clear. So using all of that information to fit a red line on that back, and then we can, do some fun stuff. So sorry. Let me let me restart. So now we isolate that red line. Right? Because Oh, I'm sorry. I totally forgot to mention this part. A lot of the fee a lot of the feedback Again, when you post on LinkedIn, people just give you free feedback. And then you guys last time gave me free feedback. So a lot of it was about posture correction or posture tracking. Yeah. Yeah. Yeah. So I'm like, okay. I'm gonna go do that. So And us as humans, when we see someone execute the exercise, we're like, oh, you know, the back is flat. The back is curved. Like, it's very straightforward evaluation. Right? So okay. Now we can do that. So I'll start with a third column, this average back shape. That's just rendering the average shape of that line during the concentric, we call it, is when you go up. Right? So then, now, I purposely arched my back as hard as I could. Yeah. Yeah. Yeah. This is a difficult data set to collect. But, purposely arching as much as I could, and what I wanted to output was, is it a good rep, a straight back rep, or is it a bad rep? And I was so pleased to see that it actually did it. You know, it's like, you you compute yeah. You compute the roundness, and if it exceeds some threshold, then it's a bad rep. So, yeah. Definitely a hard dataset to collect to, like, get myself to, like, round my back as much as I can to do that. But and I never explained the second 1, which is then you can see, like, how rounded that it is, I guess, if that makes sense. It's like an activation of how rounded that the back is. So so I proved it for the negative case, and then this video is just gonna show that for all of my good reps that it passed the test. So, yeah. I'm excited because I haven't seen anything quite like this. And when I say I haven't seen anything, when I asked ChachiPT, Claude, and Gemini to look through the literature, I haven't seen anything quite like this yet. So, yeah. Yeah. Yeah. Yeah. So so again, it's important to show that it works works for the positive case as well. And sorry, did that make sense of the this is showing the kind of the activation or like the roundness of it? And then you saw in the previous example that it was very clear that it was rounded. Yeah. That's it.