{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "convert_data_wide_to_long.ipynb",
"provenance": [],
"collapsed_sections": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KGmrniFLZ_rx"
},
"source": [
"# Convert Data from Wide to Long Format"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RIC-iofnaMyn"
},
"source": [
"`xlogit` processes data only in long format. However, converting data from wide to long format is a relatively easy task and `xlogit` provides a convinient tool for this purpose. In this tutorial, we'll explain the reshaping using two examples. Let's start by importing the `pandas` and `numpy` python packages."
]
},
{
"cell_type": "code",
"metadata": {
"id": "fueNeIAbv8sW"
},
"source": [
"import pandas as pd\n",
"import numpy as np"
],
"execution_count": 1,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "w8tpAUVvBajn"
},
"source": [
"In case you haven't installed `xlogit`, you can install it using `pip` as follows:"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "bc5G_n5yBhyL",
"outputId": "3eb7d947-2d8a-4847-e1f8-ee28613b25f8"
},
"source": [
"!pip install xlogit"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting xlogit\n",
" Downloading https://files.pythonhosted.org/packages/60/5f/9bc576d180c366af77bc04e268536e9e34be23c52a520918aa0cb56b438e/xlogit-0.1.3-py3-none-any.whl\n",
"Requirement already satisfied: scipy>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from xlogit) (1.4.1)\n",
"Requirement already satisfied: numpy>=1.13.1 in /usr/local/lib/python3.7/dist-packages (from xlogit) (1.19.5)\n",
"Installing collected packages: xlogit\n",
"Successfully installed xlogit-0.1.3\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pHFbKUIkafhX"
},
"source": [
"## Basic example"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kpaNiP4VahUp"
},
"source": [
"This example provides a quick intuition about the format conversion using `pandas`."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bLhROdletgFE"
},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-Lbu6mGro4Xl"
},
"source": [
"Let's create some artificial data in wide format for our example. Let's assume that our dataset contains choices among three transportation modes (i.e. car, train, and bus) for two individuals identified by `id`. We assume we collected the income of the individuals, and the time and cost of the transportation modes and saved the data in the `df_wide` dataframe as follows:"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 112
},
"id": "cHVYGTFjZ4km",
"outputId": "33d409ed-77e8-4816-9c4c-e47a2713ab4e"
},
"source": [
"# Create example dataset in wide format\n",
"df_wide = pd.DataFrame({'id': [1, 2],\n",
" 'income': [4, 2],\n",
" 'time_car': [2, 1],\n",
" 'time_train': [2, 4],\n",
" 'time_bus': [4, 5],\n",
" 'cost_car': [3, 7],\n",
" 'cost_train': [6, 8],\n",
" 'cost_bus': [9, 2],\n",
" 'choice': ['bus', 'car']\n",
" })\n",
"df_wide"
],
"execution_count": 3,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" id | \n",
" income | \n",
" time_car | \n",
" time_train | \n",
" time_bus | \n",
" cost_car | \n",
" cost_train | \n",
" cost_bus | \n",
" choice | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 1 | \n",
" 4 | \n",
" 2 | \n",
" 2 | \n",
" 4 | \n",
" 3 | \n",
" 6 | \n",
" 9 | \n",
" bus | \n",
"
\n",
" \n",
" | 1 | \n",
" 2 | \n",
" 2 | \n",
" 1 | \n",
" 4 | \n",
" 5 | \n",
" 7 | \n",
" 8 | \n",
" 2 | \n",
" car | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" id | \n",
" alt | \n",
" time | \n",
" cost | \n",
" income | \n",
" choice | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 1 | \n",
" bus | \n",
" 4 | \n",
" 9 | \n",
" 4 | \n",
" bus | \n",
"
\n",
" \n",
" | 1 | \n",
" 1 | \n",
" car | \n",
" 2 | \n",
" 3 | \n",
" 4 | \n",
" bus | \n",
"
\n",
" \n",
" | 2 | \n",
" 1 | \n",
" train | \n",
" 2 | \n",
" 6 | \n",
" 4 | \n",
" bus | \n",
"
\n",
" \n",
" | 3 | \n",
" 2 | \n",
" bus | \n",
" 5 | \n",
" 2 | \n",
" 2 | \n",
" car | \n",
"
\n",
" \n",
" | 4 | \n",
" 2 | \n",
" car | \n",
" 1 | \n",
" 7 | \n",
" 2 | \n",
" car | \n",
"
\n",
" \n",
" | 5 | \n",
" 2 | \n",
" train | \n",
" 4 | \n",
" 8 | \n",
" 2 | \n",
" car | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" admit | \n",
" gre | \n",
" gpa | \n",
" rank | \n",
" id | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 0.0 | \n",
" 380.0 | \n",
" 3.61 | \n",
" 3.0 | \n",
" 0 | \n",
"
\n",
" \n",
" | 1 | \n",
" 1.0 | \n",
" 660.0 | \n",
" 3.67 | \n",
" 3.0 | \n",
" 1 | \n",
"
\n",
" \n",
" | 2 | \n",
" 1.0 | \n",
" 800.0 | \n",
" 4.00 | \n",
" 1.0 | \n",
" 2 | \n",
"
\n",
" \n",
" | 3 | \n",
" 1.0 | \n",
" 640.0 | \n",
" 3.19 | \n",
" 4.0 | \n",
" 3 | \n",
"
\n",
" \n",
" | 4 | \n",
" 0.0 | \n",
" 520.0 | \n",
" 2.93 | \n",
" 4.0 | \n",
" 4 | \n",
"
\n",
" \n",
" | ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
"
\n",
" \n",
" | 395 | \n",
" 0.0 | \n",
" 620.0 | \n",
" 4.00 | \n",
" 2.0 | \n",
" 395 | \n",
"
\n",
" \n",
" | 396 | \n",
" 0.0 | \n",
" 560.0 | \n",
" 3.04 | \n",
" 3.0 | \n",
" 396 | \n",
"
\n",
" \n",
" | 397 | \n",
" 0.0 | \n",
" 460.0 | \n",
" 2.63 | \n",
" 2.0 | \n",
" 397 | \n",
"
\n",
" \n",
" | 398 | \n",
" 0.0 | \n",
" 700.0 | \n",
" 3.65 | \n",
" 2.0 | \n",
" 398 | \n",
"
\n",
" \n",
" | 399 | \n",
" 0.0 | \n",
" 600.0 | \n",
" 3.89 | \n",
" 3.0 | \n",
" 399 | \n",
"
\n",
" \n",
"
\n",
"
400 rows × 5 columns
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" id | \n",
" alt | \n",
" admit | \n",
" gre | \n",
" gpa | \n",
" rank | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 0 | \n",
" 0 | \n",
" 0.0 | \n",
" 380.0 | \n",
" 3.61 | \n",
" 3.0 | \n",
"
\n",
" \n",
" | 1 | \n",
" 0 | \n",
" 1 | \n",
" 0.0 | \n",
" 380.0 | \n",
" 3.61 | \n",
" 3.0 | \n",
"
\n",
" \n",
" | 2 | \n",
" 1 | \n",
" 0 | \n",
" 1.0 | \n",
" 660.0 | \n",
" 3.67 | \n",
" 3.0 | \n",
"
\n",
" \n",
" | 3 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 660.0 | \n",
" 3.67 | \n",
" 3.0 | \n",
"
\n",
" \n",
" | 4 | \n",
" 2 | \n",
" 0 | \n",
" 1.0 | \n",
" 800.0 | \n",
" 4.00 | \n",
" 1.0 | \n",
"
\n",
" \n",
" | ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
"
\n",
" \n",
" | 795 | \n",
" 397 | \n",
" 1 | \n",
" 0.0 | \n",
" 460.0 | \n",
" 2.63 | \n",
" 2.0 | \n",
"
\n",
" \n",
" | 796 | \n",
" 398 | \n",
" 0 | \n",
" 0.0 | \n",
" 700.0 | \n",
" 3.65 | \n",
" 2.0 | \n",
"
\n",
" \n",
" | 797 | \n",
" 398 | \n",
" 1 | \n",
" 0.0 | \n",
" 700.0 | \n",
" 3.65 | \n",
" 2.0 | \n",
"
\n",
" \n",
" | 798 | \n",
" 399 | \n",
" 0 | \n",
" 0.0 | \n",
" 600.0 | \n",
" 3.89 | \n",
" 3.0 | \n",
"
\n",
" \n",
" | 799 | \n",
" 399 | \n",
" 1 | \n",
" 0.0 | \n",
" 600.0 | \n",
" 3.89 | \n",
" 3.0 | \n",
"
\n",
" \n",
"
\n",
"
800 rows × 6 columns
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" id | \n",
" alt | \n",
" admit | \n",
" gre | \n",
" gpa | \n",
" rank | \n",
" rank_1 | \n",
" rank_2 | \n",
" rank_3 | \n",
" rank_4 | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 0 | \n",
" 0 | \n",
" 0.0 | \n",
" 380.0 | \n",
" 3.61 | \n",
" 3 | \n",
" 0 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
"
\n",
" \n",
" | 1 | \n",
" 0 | \n",
" 1 | \n",
" 0.0 | \n",
" 380.0 | \n",
" 3.61 | \n",
" 3 | \n",
" 0 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
"
\n",
" \n",
" | 2 | \n",
" 1 | \n",
" 0 | \n",
" 1.0 | \n",
" 660.0 | \n",
" 3.67 | \n",
" 3 | \n",
" 0 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
"
\n",
" \n",
" | 3 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 660.0 | \n",
" 3.67 | \n",
" 3 | \n",
" 0 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
"
\n",
" \n",
" | 4 | \n",
" 2 | \n",
" 0 | \n",
" 1.0 | \n",
" 800.0 | \n",
" 4.00 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 0 | \n",
" 0 | \n",
"
\n",
" \n",
" | ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
" ... | \n",
"
\n",
" \n",
" | 795 | \n",
" 397 | \n",
" 1 | \n",
" 0.0 | \n",
" 460.0 | \n",
" 2.63 | \n",
" 2 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
" 0 | \n",
"
\n",
" \n",
" | 796 | \n",
" 398 | \n",
" 0 | \n",
" 0.0 | \n",
" 700.0 | \n",
" 3.65 | \n",
" 2 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
" 0 | \n",
"
\n",
" \n",
" | 797 | \n",
" 398 | \n",
" 1 | \n",
" 0.0 | \n",
" 700.0 | \n",
" 3.65 | \n",
" 2 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
" 0 | \n",
"
\n",
" \n",
" | 798 | \n",
" 399 | \n",
" 0 | \n",
" 0.0 | \n",
" 600.0 | \n",
" 3.89 | \n",
" 3 | \n",
" 0 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
"
\n",
" \n",
" | 799 | \n",
" 399 | \n",
" 1 | \n",
" 0.0 | \n",
" 600.0 | \n",
" 3.89 | \n",
" 3 | \n",
" 0 | \n",
" 0 | \n",
" 1 | \n",
" 0 | \n",
"
\n",
" \n",
"
\n",
"
800 rows × 10 columns
\n",
"