Rooting MCU1

From Unofficial Tesla Tech
Revision as of 15:56, 14 February 2020 by Carl (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Foreward

Ok, you've gotten access to your eMMC chip and it is time to modify the filesystem to root it. This post will describe AP1 cars and all earlier non-AP cars, as I've had to figure this all out myself and my car is an AP1 car. I welcome all comments and corrections, although my time is very limited so I can't spoon-feed or hold hands.

There are methods of rooting which don't involve soldering of course, through software vulns. (depending on firmware version) But these are always very difficult to find, and always very easy to patch, so are carefully guarded. These are not my work and I have no right to release them, and would never do so in any case. If you'd rather this then please prepare your $1k+ for your (unfriendly) neighborhood commercial rooter (Ingineer, wk057), and give up any idea of having privacy and root for yourself. My method can never be patched.

As a reminder to my prior post, the CID is the daughterboard on the MCU mainboard. The CID is made by nVidia for Tesla, and although it seems to be an MMX formfactor (of the Qseven persuasion), it is not, it is VCM, a special OEM custom for Tesla. A check of the 5v pinouts (on the opposite end) will confirm this. (Don't ask me how I know...)

As it's made by nVidia, they've used the typical system on their high-end video cards, that is a firmware update goes alternately to partition 1 or 2 whichever is not currently active, the new firmware is verified as a good write, then the car reboots to the new firmware and deploys staged components to the ECMs in the rest of the car.

A boot coprocessor lives in the Tegra 3 chip, distinct from the actual T3 processor, and on reset this coprocessor initializes to the first address in the Spansion chip on the obverse of the CID. You'll notice that this is a rather large-capacity chip for an embedded device (512MB), and the reason is that it keeps track of which partition in the eMMC is the active one, and then builds the OS from that in RAM, at each boot. Once complete, the coprocessor chain-boots to the T3 processor, which boots to the filesystem in RAM, and mounts eMMC partition 3 as /var and 4 as /home. I strongly suspect the coprocessor and Spansion chip to be the Gateway subsystem.

So it's likely that partitions 1 and 2 will have different versions of the firmware. (older, and newer) Everything is checked real-time by the boot coprocessor using code-signing (started very early in its init process in what is effectively its BIOS), and so it is very important that the active eMMC partition's firmware match the version in the Spansion chip. If you unilaterally upgrade the active partition to a different version, or change the eMMC chip to one from another CID, boot will fail (due to code-signing) and black screen. (Don't ask me how I know...)

When people take their car to Tesla for a fix to black screen or rebooting of the MCU, Tesla often 'magically' fixes it quickly and cheaply... they are simply putting the customer on the other partition, kicking the eMMC write failure (can) down the road. Rest assured, it will be back, soon.

Firmware updates must only be written to the inactive partition. Then you will deploy (using a Tesla script), the system will determine what kind of car you have and stage the correct modules (from 1or2/deploy/seed_artifacts_v2 which holds all modules for all cars) to 4/cid-updater/gateway-staging-work/gwrelease*, which holds only those ECM ('Electronic Control Module') files which are specific to your car. Once staged, the update clock will come up on your MCU and you choose when to initiate it.

I wouldn’t normally expect the staged modules to be in /home. (Note to Tesla: Under Posix standards, /var/spool is the proper place for stuff like this, not /home) The manifest file there holds the name and version of each .hex file, in the order that they should be applied.

First Step -- BACK UP!!

If you haven't already, dd you the whole original eMMC image off the chip and then individual partitions, and set aside as your Golden ones, as I'd described in the earlier thread.

Second Step -- Reconnaissance

It is important to know which is the current active part, 1 or 2.

Now the nice thing with the chip out is you can mount partitions 1 and 2 with no problem, even though they are squashfs. They're even read/write.

# mount -o loop /dev/mmcblk0p1 /media/1
# mount -o loop /dev/mmcblk0p2 /media/2
# mount /dev/mmcblk0p3 /media/3
# mount /dev/mmcblk0p4 /media/4

(Needless to say, create your mount dirs under /media first)

Well if your CID is not booted, you can't tell whether part 1 or part 2 is booted. So put in flag files:

# touch /media/1/1
# touch /media/2/2

... then later see which one ends up in /usr after boot. Of course this flag file will be overwritten on the next firmware update, but you can recreate it on the eMMC since you will have root. (Hint: mount -o remount,rw)

100px Safety Glasses Warning

It's important to note that anytime you are working as root you can easily damage your filesystem, to uselessness. So remain conscious and don't work on this drunk.

Now to our filesystem. Interesting directories and files, IMHO:

  • 1-2/deploy/ - the complete update mechanism for the car. boot.img is there, and all it should take is a dd of that to the 512MB flash chip. (If you can get to it) version_map2.tsv contains all the file versions for different configs of cars, for example RearDrive type ?, and 4WD type ?, etc. So the reason for staging files from here is to sort out the only ones which are applicable to this particular car.
  • 1-2/etc/openvpn/teslavpn.conf.prod
  • 1-2/local/bin/
  • 1-2/local/bin/get-vitals
  • 1-2/share/openssl-blacklist, openvpn, openvpn-blacklist
  • 1-2/tesla/UI/
  • 3/spool/cron/crontabs/root
  • 4/tesla/.crashlogs/QtCar-crash-*.txt
  • 1-2/local/bin/gwxfer perl script allows transfer of the gateway config from gw to cid filesystem and back.
  • Templates of the openvpn config files (teslavpn.conf.dev and
  • teslavpn.conf.prod) in 1-2/etc/openvpn folder.
  • 3 on-screen browser config and cache files, .bashrc and friends, my radio config with my internet and OTA stations (SQLite database), crashlogs (187 files!), QtCar, QtCarGpsManager, QtCarMediaServerV2, QtCarNetManager, QtCarParrot, QtCarServer -yikes-, QtCarSpeedRecognizer, QtCarVehicle -yikes-, and unknown, gstreamer config files, .ssh (with only authorized_keys), and cached map tiles of my area (http & https .cache type).

Notice that there is an /etc directory in partitions 1 and 2, but the only things in it are a log mask file for the Sierra GSM card, and an openvpn directory — hello. In it are:

  • a script that updates resolv.conf on connexion of VPN, and
  • the openvpn.conf file, which points to the car’s cert, car key (a cert), the CA cert, and the (depreciated) TLS remote option to authorize the remote host. (which is who we might expect) The certs belong in this directory but are not in partition 1 or 2, likely added when the OS is built in RAM since they are car-specific.

Where is passwd? Oh, in 1(&2)/share/base-passwd/passwd.master and also there is group.master. Nothing special here, just the usual.

So remember, the basics of the VPN and the passwd file are in partitions 1 and 2 which get overwritten at updates. IOW you can not add users nor VPNs without special cron scripts.

Where is shadow? A quick mlocate and: 3/etc/. Ok this is good news, it's in a non-volatile partition so we can modify passwords etc, and our work will be remembered in perpetuum. Also there are birthday, car keys, vin, gateway.cfg, etc, and openvpn which contains the certs we’ve learned about.

Third Step -- Rooting

Does shadow look sane? Yup, sure does, so let’s look at how we can adjust it.

100px CAUTION - Science Content

~~ About Linux Passwords ~~

Why is there a-such-a thing as a shadow file anyway, when there’s /etc/passwd? Can’t we just set passwd to root access rights only? No because passwd has other things in it that non-privileged accounts need access to, like which shell to use, where the home directory is, etc. passwd must be world-readable (although not world-writable of course). shadow holds the info about users that can be isolated to root only (which handles logins). Now what does a typical shadow entry look like anyway? root:!$6$Ud8AO7B9kOFvITQm$M8EikCPKFh6iFHfLz6vjM5IzBP211qyPxe/48Nt bjOxpZtTEQYIAzDAiiVrtHlWgQaTZk2AV6x6VUK76BnGf8/:15926:0:99999:7:::

  • root: Username.
  • ! Disable user! Well that’s just not on, is it? To disable root?
  • $6$ The password was hashed using the SHA512 algorithm. ($ is just a delimiter)
  • Ud8AO7B9kOFvITQm$ The ‘salt‘ used in hashing the password, chosen randomly at each password hashing.
  • M8EikCPKFh6iFHfLz6vjM5IzBP211qyPxe/48NtbjOx
pZtTEQYIAzDAiiVrtHlWgQaTZk2AV6x6VUK76BnGf8/ The actual password result after it’s passed through the one-way SHA512 hashing algorithm. This can not be reversed out to the password with today’s computers, it can only be brute-forced by trying every combination of letters, numbers, and special characters, up to however many characters the password may have. This could take hundreds of thousands of years.
  • 15926 Days since Jan 1, 1970 that the password was last changed.
  • 0 Maximum number of days the password is valid. (0 is infinite)
  • 99999 Number of days before password is to expire that user is warned.
  • 7 Number of days after password expires that the account is disabled.
  • … and so on.


All we really care about for this is username-password. We know at this point that partition 3/etc is swept up into the filesystem built in RAM (probably symlinked), and that it is non-volatile, so we can make changes to passwords here and get away with it. But we can’t add users as the mechanism that users are created and tended in this filesystem construct is in parts 1 or 2, so let’s not change usernames or we might break something.

To create our own personal passwords, there are lots of hard ways and thousands of wrong ways, but the easy way to create a new SHA512 salt & hash is with:

# python3 -c “from getpass import getpass; from crypt import *; p=getpass(); print(‘\n’+crypt(p, METHOD_SHA512)) if p==getpass(‘Please repeat: ‘) else print(‘\nFailed repeating.’)”

Just put the result into the shadow file in place of the existing values using your favorite text editor {*cough*nano*}, changing root’s and tesla's password. I don't recommend changing nvidia’s password, nor tesla1 and 2 if you ever want Tesla to contact the car again. (for some reason ... so they can determine that you are rooted and blacklist you if not already) Now remove that specious exclamation mark. DO NOT change anything else in the shadow file.

It's Ok to change the password in this manner because there is no system booted to it. Otherwise if booted use only the standard tools, useradd, usermod, userdel, groupadd, etc.

Fourth Step -- Retaining Root, Customizing

Alright, we now have our passwords set, so how can we further ensure access? It’s quite a risky operation to get this chip un/resoldered so we want to make sure of success. Let’s insert our SSH cert, since we’ll be logging in by SSH. Through experimentation I’ve found that a Tesla daemon inconsiderately overwrites SSH certs periodically so we’ll set up a cron job which executes our custom script.

Let’s put our custom work in a /home directory; we can’t put it in partitions 1 or 2 of the eMMC since those are the OS, updated by Tesla on a regular basis. Remember partition 3 is mounted /var and 4 is /home, so both are non-volatile. The only user directory in partition 4 is tesla, so let’s create our subdir there named, say, .local (Note: the leading dot). In that dir, to wit:

Master

# nano start.sh

#!/bin/sh
# Get the current epoch
epoch=`date +%s`
# Dont run until epoch is over 1447400000, or time on boot.
while [ ! -d /var/run/ ] || [ ${epoch} -le 1447400000 ]
do
epoch=`date +%s`
    echo ‘start.sh: waiting for system to start’
done
 # Variables
wk_path=’/home/tesla/.local’
atomic_path=’/var/run/.local’
 if [ -e ${wk_path}/customOps ]; then
    bash ${wk_path}/customOps &
fi
 # Run the satellite scripts once. Looping is in each script as needed.
if mkdir ${atomic_path}
then
    sh ${wk_path}/log >/dev/null 2>&1 &
    sh ${wk_path}/ssh >/dev/null 2>&1 &
    sh ${wk_path}/root >/dev/null 2>&1 &
    sh ${wk_path}/diag >/dev/null 2>&1 &
    sh ${wk_path}/proxy >/dev/null 2>&1 &
    sh ${wk_path}/autopilot >/dev/null 2>&1 &
fi

Note: When calling a shell script (#!/bin/sh), we all know better than to use ‘bash’ instead of sh don’t we? We’re not farmers.

This is our master script which calls sub-functions we create to do various things.

Now we're going to create the sub-scripts which the master script (driven by cron), calls. Needless to say, these are my solutions. You can add or subtract as you like. Also, I'm running firmware v2017.42 and some of these functions may no longer work with newer firmware.

customOps

# Stuff you might like to do temporarily or which may not deserve its own sub-script
#while [ 1 -eq 1 ]
#do

# Loop waiting one minute between cycles
#/bin/sleep 60
#done

log

# Disable all logging to save eMMC
service rsyslog stop >/dev/null 2>&1

ssh

]# Initiate an reverse ssh tunnel to your home server as a backup.
# Put your SSH private and public keys in 4/tesla/.local/.ssh/ and create a 'tesla' user on your home server with this private key.
# ssh-keygen -t rsa -o -b 4096 -a 100 -v        # Yes, enter a good password.

# Determine a remote port based on the last five digits of your VIN. (Be sure to open that port incoming on your home server)
port=5$(cut -c 13-17 < /var/etc/vin)

# Path to the ssh keys
id_path='/home/tesla/.local/.ssh'


# Initialize PID
PID=''

# Once started, loop indefinitely while the car is started.
while [ 1 -eq 1 ]
do
# If a running ssh is found, don't create another outbound ssh connection.
    if [ -z $PID ] || ! ps -p $PID > /dev/null
    then
        # Create the outbound ssh connection using a public ssh key.
        # Allow all host keys and be intelligent about checking if the
        # connection is up. Terminate the session if the server doesn't
        # respond in 3 x 15 seconds.
        /usr/bin/ssh -p55522 -N -R $port:localhost:22 -o UserKnownHostsFile=/dev/null \
        -o StrictHostKeyChecking=no -o ServerAliveInterval=15 -o ExitOnForwardFailure=yes \
        -i $id_path/id_rsa tesla@{yourServerIP} >/dev/null 2>&1 &
        PID=$!
    fi

    # Sleep 30 seconds before retrying the ssh connection.
    /bin/sleep 30
done

root

# Let's let ourselves into the CID as root.
# It would be a good idea to create different keypairs for the above outbound reverse tunnel, and for this access, private keys on respective sides.
# ssh-keygen -t rsa -o -b 4096 -a 100 -v

mkdir -p -m 0700 /root/.ssh

/bin/cat 'ssh-rsa {yourPublicKey}' > /root/.ssh/authorized_keys

/bin/chmod 0600 /root/.ssh/authorized_keys

diag

# Enable the diagnostic ethernet port by manipulating seceth.
# touch a file 'enableDiag if you want this to work.

path='/home/tesla/.local'

while [ 1 -eq 1 ]
do
# If enableDiag exists in our path, create a fw rule and loop the port open cmd.
if [ -e ${path}/enableDiag ]
then
    # Local variables
    GTW_LOCK_PERIOD=30
    SEND_INTERVAL=$(($GTW_LOCK_PERIOD/3))
    SALT="pr8d+VGYxcuRex3cg3MmrO8Sk6MHxQoBlVbd"
    SACCESS=/var/etc/saccess
    SECETH_PORT=18466
    VIN=`cat /var/etc/vin`

    # Block NONCE traffic from toolbox so that it doesn't cause the diag port to close.
    /sbin/iptables -I INPUT -p udp --dport 18466 -m string --algo bm --string 'req' -j DROP

    # Create an infinite while loop that keeps the diagnostic port open.
    while [ -e ${path}/enableDiag ]
    do
        # Get the latest token and generate ether key.
        TOKEN=`cat $SACCESS/tesla1`
        ETHER_KEY=`echo -n "${SALT}${VIN}${TOKEN}" | sha1sum | cut -b 1-16`

        # Send the generated ether key to seceth so it will open the diag port.
        printf "key $ETHER_KEY\x0a" | \
        socat -u - udp-sendto:localhost:$SECETH_PORT

        # Gateway has a 30 second lock period. Diag port will close if we do not loop quickly
        /bin/sleep $SEND_INTERVAL
    done

    # Remove NONCE firewall block after enableDiag file is removed.
    /sbin/iptables -D INPUT -p udp --dport 18466 -m string --algo bm --string 'req' -j DROP
fi

# Loop waiting one minute between cycles until enableDiag exists
/bin/sleep 60

done

proxy

# We're going to be interlocutor to make sure Tesla behaves.

# Debug mode
#set -x

# Initial variables.
pid=0

while [ 1 -eq 1 ]
do
    # If process is not running, start it and store the PID.
    if ! ps -$pid >/dev/null 2>&1
    then
        # Run socat in the background to proxy queries.
        socat tcp4-listen:4567,reuseaddr,fork tcp4:mothership.vn.teslamotors.com:4567 >/dev/null 2>&1 &
        
        # Store the PID of the last running command to be checked on next loop.
        pid=$!
    fi
    
    # Sleep for 60 seconds between each loop.
    /bin/sleep 60
done

exit 0

autopilot

while [ 1 -eq 1 ]
do
/usr/local/bin/sdv GUI_disableAutosteerRestrictions true
/usr/local/bin/sdv GUI_handsOnRequirementDisable true
# Loop waiting one minute between cycles
/bin/sleep 60
done

Now it's time to cause cron to trigger the master script. If logged in to the CID create /var/spool/cron/crontabs/root with:

# EDITOR=nano crontab -e

Or, if you have the chip mounted in a chip-carrier just create 3/spool/cron/crontabs/root (using nano) and

        SHELL=/bin/sh
        @reboot  sleep 60; /sbin/iptables -I INPUT -i tun0 -p tcp -m tcp --dport 22 -j DROP; /sbin/iptables -I INPUT -i tun0 -p tcp -m tcp --dport 25956 -j DROP; /sbin/iptables -I INPUT -i parrot -j ACCEPT; /home/tesla/.local/start.sh

# Want Developer mode?  Enable this:
       */15 * * * * curl "http://cid:4035/set_data_value?name=GUI_developerMode&value=true"

# Want Diagnostic mode?  Enable this:
         */15 * * * * curl "http://cid:4035/set_data_value?name=GUI_diagnosticMode&value=true"

# chmod +x start.sh customOps log ssh root diag proxy autopilot

And just to be safer, save 3/etc/birthday, carkeys.tar, gateway.cfg, mcu_commands_key, tesla.env, vdt-config.json, and vin.

Congratulations, you have now rooted your car. Make sure you have everything exactly right, umount properly, and have the chip soldered back on.

As to VPN you might like to mess with OpenVPN which is installed, but I'm setting up WireGuard if I ever get time.

Carl A. Cook