{"cells":[{"cell_type":"markdown","metadata":{"id":"OD2g9xsgxSYy"},"source":["# Guide to Extracting Data w/ APIs from the Withings Body+ Scale\n","\n","
\n","\n","Are smart TVs, smartphones, or smart watches not enough? Introducing the [Withings Body+ Scale](https://www.withings.com/us/en/body-plus), a smart scale that can not only weigh you, but also leverage its Wi-Fi capabilities to enable you to store your historical weight data for your viewing pleasure. While this notebook is meant for the Body+ Scale, it can be easily adapted to any other Withings scale with minor modifications.\n","\n","Also note that you need to set up your Withings Body+ Scale prior to running this notebook. To do this, simply follow the instructions on the app to (1) pair your phone to your scale and (2) connect your scale to your home internet.\n","\n","If you want to know more about Withings Body+ Scale, see the [README](https://github.com/alrojo/wearipedia/tree/main/wearables/withings-scale-bodyplus) for a detailed analysis of performances, sensors, data privacy, and extraction pipelines.\n","\n","The API makes the following parameters available:\n","\n","Parameter Name | Sampling Frequency \n","-------------------|------------------\n","Weight (kg) | Per weighing\n","Fat Free Mass (kg) | Per weighing\n","Fat Ratio (%) | Per weighing\n","Fat Mass Weight (kg) | Per weighing\n","Muscle Mass (kg) | Per weighing\n","Hydration (kg) | Per weighing\n","Bone Mass (kg) | Per weighing\n","\n","\n","
\n","In this guide, we sequentially cover the following **five** topics to extract from the Withings API:\n","1. **Setup**\n"," - 1.1: We include some details about what a clinical study participant may be expected to do, assuming you use the public API as we did.\n","2. **Authentication/Authorization**\n"," - This requires a couple extra steps on your part\n","3. **Data extraction**\n"," - Due to our own limited sample size, which precludes long time horizon data analysis, we include a data extraction step that loads an artificial dataset we randomly generate.\n","4. **Data visualization**\n"," - 4.1: We look at trends in both weight and fat (%) over time (months).\n"," - 4.2: We also look at how weight varies over the time of day and downsample weight to a single measurement per day.\n","5. **Data analysis**\n"," - We aggregate body weight by time of day, then conduct a brief statistical analysis to check whether body weight indeed increases after a meal.\n","\n","\n","Disclaimer: this notebook is purely for educational purposes. All of the data currently stored in this notebook is purely *synthetic*, meaning randomly generated according to rules we created. Despite this, the end-to-end data extraction pipeline has been tested on our own data, meaning that if you enter your own credentials on your own Colab instance, you can visualize your own *real* data. That being said, we were unable to thoroughly test the timezone functionality, though, since we only have one account, so beware."]},{"cell_type":"markdown","metadata":{"id":"EGZZcSOH3PT1"},"source":["# 1. Setup\n","\n","## 1.1 Study participant setup and usage\n","\n","To set up the scale itself so that you can run this notebook, download the Withings app and follow the instructions on the app. The app will pair via bluetooth to the scale, creating an interface to connect the scale to Wi-Fi. In our experience, this process was frought with difficulties.\n","\n","Note that the Wi-Fi network must have no captive portal, only simple username / password authentication. This means that connecting the scale to Wi-Fi may be challenging on a university campus setting, where Wi-Fi is hidden behind layers of authentication. One potential workaround is to view the serial number (which is the same as the MAC address) and register the MAC address through the IT department, which should allow a connection to university campus Wi-Fi.\n","\n","## 1.2 Library import\n","Relevant libraries are imported below. Just run the code to import all the libraries."]},{"cell_type":"code","source":["#@title Library import + setup\n","\n","# sneak in the library imports etc. here :)\n","\n","# We randomly generate a large artificial dataset to make our demo analysis\n","# more interesting and to avoid using real data. This is not needed for\n","# data extraction.\n","!rm -f random_data.csv && wget -q https://gist.githubusercontent.com/stanford-health-wearables/3e5bdd4dfc06a4290038fabf34732ca3/raw/c99a50c1d943903c867364dc6c9a11d83fb4e42a/random_data.csv\n","\n","# upgrade scipy, since we need \"intercept_stderr\"\n","# from https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.linregress.html\n","!pip install --upgrade scipy -q\n","\n","import requests\n","import urllib\n","import json\n","from datetime import datetime\n","\n","import numpy as np\n","import pandas as pd\n","import matplotlib.pyplot as plt\n","import seaborn as sns\n","from scipy.ndimage import gaussian_filter\n","from scipy import stats"],"metadata":{"cellView":"form","id":"C2Xoi3ci9bVF"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"d2Kbo6d4waj9"},"source":["# 2. Authentication/Authorization\n","\n","To be able to make requests to the API, the easiest way is to use the public developer API. This section roughly follows the steps outlined [here](https://developer.dexcom.com/authentication) on their website.\n","\n","First, follow the non-colab steps listed below:\n","\n","1. Visit the [developer portal](https://developer.withings.com/) and click \"Open Developer Dashboard\" on the top right.\n","2. Once logged in, click \"Add an app\".\n","3. For now, you can just click \"I don't know\" under \"Services\", accept terms of use, and click \"Next\".\n","5. Put whatever you want under \"Application Name\" (we used `withings-test`), anything under \"Application Description\", and \"https://wbsapi.withings.net/v2/oauth2\" under Registered URLs, then click \"Done\".\n"," - NOTE: \"registered URLs\" is intended to be a URL to a webserver you control and can receive requests from. However, in this notebook we are simply using it as a placeholder, as this functionality is not strictly necessary for obtaining your data.\n","\n","In the end, you should see something like the below.\n","\n","
\n","\n","Now we can proceed with the rest of the notebook.\n","\n","To be able to make requests to the API and extract the data we need, we need to first issue an access token. This (ephemeral) access token will serve as our key to the data. While, you don't necessarily need to be familiar with how the issuing of the authtoken occurs, you can learn more about it by visiting [the official Withings tutorial](https://developer.withings.com/developer-guide/v3/integration-guide/public-health-data-api/get-access/oauth-web-flow/)."]},{"cell_type":"code","execution_count":null,"metadata":{"cellView":"form","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":15,"status":"ok","timestamp":1662923875853,"user":{"displayName":"Rodrigo J Castellon","userId":"11221204973290737308"},"user_tz":240},"id":"ev3-mu5ou88c","outputId":"57b6c88d-a7e4-4e47-b407-24a1e4a74493"},"outputs":[{"output_type":"stream","name":"stdout","text":["https://account.withings.com/oauth2_user/authorize2?response_type=code&client_id=&state=string&scope=user.info,user.metrics&redirect_uri=https://wbsapi.withings.net/v2/oauth2\n"]}],"source":["#@title 6. Enter your credentials below (from the application you just created)\n","CLIENT_ID = \"\" #@param {type:\"string\"}\n","CUSTOMER_SECRET = \"\" #@param {type:\"string\"}\n","\n","STATE = 'string'\n","ACCOUNT_URL = 'https://account.withings.com'\n","CALLBACK_URI = 'https://wbsapi.withings.net/v2/oauth2'\n","\n","\n","payload = {'response_type': 'code', # imposed string by the api\n"," 'client_id': CLIENT_ID,\n"," 'state': STATE,\n"," 'scope': 'user.info,user.metrics', # see docs for enhanced scope\n"," 'redirect_uri': CALLBACK_URI, # URL of this app\n"," #'mode': 'demo' # Use demo mode, DELETE THIS FOR REAL APP\n","}\n","\n","url = f'{ACCOUNT_URL}/oauth2_user/authorize2?'\n","\n","for key, value in payload.items():\n"," url += f'{key}={value}&'\n","\n","url = url[:-1]\n","\n","print(url)"]},{"cell_type":"markdown","metadata":{"id":"kpLZcG86vSuL"},"source":["7. Now visit the above URL and click \"Allow this app\", and copy the URL you were redirected to into the text field below. Note that if you mess up once, you have to go through the above URL again (including clicking \"Allow this app\"). Also, the URL is only valid for 30 seconds, so be quick!"]},{"cell_type":"code","execution_count":null,"metadata":{"cellView":"form","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1996,"status":"ok","timestamp":1662923877845,"user":{"displayName":"Rodrigo J Castellon","userId":"11221204973290737308"},"user_tz":240},"id":"oZ8kzSuBoily","outputId":"0fbb67d1-ebb5-4009-c2ea-a7677f2afbde"},"outputs":[{"output_type":"stream","name":"stdout","text":["Took too long to paste in redirect URL. Please repeat step 7.\n"]}],"source":["#@title 7. Copy and paste the URL you were redirected to below\n","redirect_url = \"https://wbsapi.withings.net/v2/oauth2?code=e46275ce54994fa6e6eadc50cd4dd45467c01e0e&state=string\" #@param {type:\"string\"}\n","\n","try:\n"," code = urllib.parse.parse_qs(urllib.parse.urlparse(redirect_url).query)['code'][0]\n","except Exception as e:\n"," print(f'Caught error:\\n{e}\\n')\n"," print(\"Please copy and paste the entire URL (including https)\")\n","\n","params = {\n"," 'action': 'requesttoken',\n"," 'grant_type': 'authorization_code',\n"," 'client_id': CLIENT_ID,\n"," 'client_secret': CUSTOMER_SECRET,\n"," 'code': code,\n"," #'scope': 'user.info',\n"," 'redirect_uri': 'https://wbsapi.withings.net/v2/oauth2'\n","}\n","\n","out = requests.get('https://wbsapi.withings.net/v2/oauth2', data=params)\n","\n","out = json.loads(out.text)\n","\n","try:\n"," access_token = out['body']['access_token']\n","except KeyError as e:\n"," print('Took too long to paste in redirect URL. Please repeat step 7.')"]},{"cell_type":"markdown","metadata":{"id":"JObtRZuFvuJl"},"source":["Now that we have our access token, we can begin making requests to the API! This access token will last only three hours, though, so you would need to re-do step 7 if three hours pass."]},{"cell_type":"markdown","metadata":{"id":"GUtc7x14ebLe"},"source":["# 3. Data extraction\n","\n","Here, data extraction is pretty simple! All we need to do is make a GET request with the right query parameters. See [the overall health data API page](https://developer.withings.com/developer-guide/v3/integration-guide/public-health-data-api/data-api/all-available-health-data) or the [\"Measure\" endpoints specifically](https://developer.withings.com/api-reference#operation/measure-getmeas) for more info, if you want to change up these query parameters for your own usage.\n","\n","Below you can uncheck the \"use_synthetic\" box to instead use your own *real* data via Withings API calls!"]},{"cell_type":"code","execution_count":null,"metadata":{"cellView":"form","colab":{"base_uri":"https://localhost:8080/","height":424},"executionInfo":{"elapsed":4,"status":"ok","timestamp":1662923877845,"user":{"displayName":"Rodrigo J Castellon","userId":"11221204973290737308"},"user_tz":240},"id":"oIhFUppKKwBC","outputId":"73e4e624-cd78-40ca-d67f-2f794372bf09"},"outputs":[{"output_type":"execute_result","data":{"text/plain":[" date Weight (kg) Fat Ratio (%)\n","0 2020-01-01 16:00:20.835392 68.281531 27.600354\n","1 2020-01-01 10:49:38.738726 68.798625 29.339849\n","2 2020-01-02 20:20:00.887639 68.354385 28.474596\n","3 2020-01-02 14:49:07.478133 68.778135 29.911508\n","4 2020-01-02 23:45:25.764174 66.428940 28.372731\n","... ... ... ...\n","1375 2022-05-14 19:18:22.484972 62.143377 17.195181\n","1376 2022-05-15 12:03:44.424457 60.446983 14.965986\n","1377 2022-05-16 20:46:23.244587 57.514826 15.528326\n","1378 2022-05-16 21:08:47.633776 59.242820 14.460136\n","1379 2022-05-18 11:34:03.277528 62.786334 16.972130\n","\n","[1380 rows x 3 columns]"],"text/html":["\n","
| \n"," | date | \n","Weight (kg) | \n","Fat Ratio (%) | \n","
|---|---|---|---|
| 0 | \n","2020-01-01 16:00:20.835392 | \n","68.281531 | \n","27.600354 | \n","
| 1 | \n","2020-01-01 10:49:38.738726 | \n","68.798625 | \n","29.339849 | \n","
| 2 | \n","2020-01-02 20:20:00.887639 | \n","68.354385 | \n","28.474596 | \n","
| 3 | \n","2020-01-02 14:49:07.478133 | \n","68.778135 | \n","29.911508 | \n","
| 4 | \n","2020-01-02 23:45:25.764174 | \n","66.428940 | \n","28.372731 | \n","
| ... | \n","... | \n","... | \n","... | \n","
| 1375 | \n","2022-05-14 19:18:22.484972 | \n","62.143377 | \n","17.195181 | \n","
| 1376 | \n","2022-05-15 12:03:44.424457 | \n","60.446983 | \n","14.965986 | \n","
| 1377 | \n","2022-05-16 20:46:23.244587 | \n","57.514826 | \n","15.528326 | \n","
| 1378 | \n","2022-05-16 21:08:47.633776 | \n","59.242820 | \n","14.460136 | \n","
| 1379 | \n","2022-05-18 11:34:03.277528 | \n","62.786334 | \n","16.972130 | \n","
1380 rows × 3 columns
\n","| \n"," | date | \n","Weight (kg) | \n","
|---|---|---|
| 0 | \n","2020-01-01 | \n","69.003367 | \n","
| 1 | \n","2020-01-02 | \n","67.755449 | \n","
| 2 | \n","2020-01-03 | \n","66.599716 | \n","
| 3 | \n","2020-01-04 | \n","68.204664 | \n","
| 4 | \n","2020-01-05 | \n","69.622622 | \n","
| ... | \n","... | \n","... | \n","
| 647 | \n","2022-05-12 | \n","58.039862 | \n","
| 648 | \n","2022-05-13 | \n","59.494033 | \n","
| 649 | \n","2022-05-14 | \n","60.823615 | \n","
| 650 | \n","2022-05-15 | \n","60.030781 | \n","
| 651 | \n","2022-05-16 | \n","57.698450 | \n","
652 rows × 2 columns
\n","