Upload Growatt Data to PVoutput with Raspberry Pi

This post is about uploading the data from my Growatt Solar Inverter to pvoutput.org using a dedicated Raspberry Pi.

Initially I found this script somewhere on the internet. It’s written by Sander Plug, according to the code. So all credits go to Sander.
But since the original page is not available anymore I decided to make this manual. ~~https://123zonne-energie.ning.com/profiles/blogs/growatt-wifi-module-via-raspberry-pi-automatische-upload-naar~~
EDIT: 2022-june-20: For Raspian/Debian “Bullseye” installation a few extra modifications are required to make things work. These settings are modified in the post below.

Step 1. Install a Raspberry Pi using Rasbian

I Use a RaspBerry Pi model 1B. This is one of the oldest models. But since there is no user interaction this model is powerfull enough for this purpose. If you have a newer model it will do as well of course.

This is not a manual to install Rasbian. But basically:

Download Rasbian from https://www.raspberrypi.org/downloads/raspbian/
- I used the Buster Lite edition
Use Win32DiskImager to put Rasbian to an SD card.
After reinserting the SD card to your computer put an empty file called SSH on the FAT32 partition. This will make Rasbian in Headless mode. Thus, the SSH-server is started and accessible. No need attaching a monitor/keyboard.
Attach the raspberry pi to the network. Insert the SD card and power.
I used a DHCP reservation. Otherwise to need to determine the IP. I advise a DHCP reservation anyway since the IP need to be fixed for the GroWatt solution
Connect to the Raspberry Pi through SSH (e.g. using Putty)
- Standard username “pi” with password “raspberry”
Fire up raspi-config to make some basic settings (sudo raspi-config)
- Change standard password
- Change hostname under “Network Options”
- Change Regional settings and timezone to match yours
- Expand filesystem under “Advanced” to make use of the entire SD card
- Reboot the pi (sudo reboot)

Step 2. Enable portforwarding

In this step we will forward all traffic for TCP port 5279 to server.growatt.com. By doing this all traffic from your Growatt inverter will still be forwarded to Growatt, so you will still be able to use the original Growatt data collection services.

Enable port forwarding:
- sudo nano /proc/sys/net/ipv4/ip_forward
- replace the 0 with a 1
- Raspbian: Debian “Bullseye”
  - sysctl -w net.ipv4.ip_forward=1
  - sudo nano /etc/sysctl.conf
    - Add the following lines
    - net.ipv4.ip_forward = 1
  - sudo nano /etc/sysctl.d/99-ipv.conf
  - Add the following lines
    - net.ipv4.ip_forward = 1
    - net.ipv6.conf.all.forwarding = 1
  - then run
    - sudo systemctl restart procps
    - sudo sysctl -p
Determine the IP adress for server.growatt.com. At this moment this is 47.91.67.66 but it may vary over time.
Run the following commands on the Pi.
- sudo iptables -t nat -A PREROUTING -p tcp --dport 5279 -j DNAT --to-destination 47.91.67.66:5279
- sudo iptables -t nat -A POSTROUTING -j MASQUERADE
- sudo iptables -t nat -L
- sudo bash -c 'iptables-save > /etc/network/iptables'
Edit the file: /etc/network/interfaces
- sudo nano /etc/network/interfaces
- Put the follwing line at the end of the file:
  - pre-up iptables-restore < /etc/network/iptables
Finally we have to edit the “/etc/sysctl.conf” file to make it work accross reboots
- sudo nano /etc/sysctl.conf
- Uncomment the next line to enable packet forwarding for IPv4
  - net.ipv4.ip_forward=1
- Put the follwing line at the end of the file:
  - net.ipv6.conf.all.disable_ipv6 = 1
sudo nano /etc/rc.local
- Put the following lines BEFORE the exit 0 line
  - sudo iptables -t nat -A PREROUTING -p tcp --dport 5279 -j DNAT --to-destination 47.91.67.66:5279
  - sudo iptables -t nat -A POSTROUTING -j MASQUERADE
  - sudo bash /home/pvoutput/scripts/process_growatt_pvoutput.sh
Reboot the pi (sudo reboot)

Step 3. Alter the Growatt Wifi Module

The Growatt Wifi Module is by default configured to communicate direct to server.growatt.com. This needs to be changed to so that the Growatt Wifi Module points to the (internal) IP address of the Raspberry Pi. The forward rules will ensure that from there the traffic is re-routed to the server.growatt.com servers as normal.

Browse to the IP adress of the Growat Wifi Module
http://<GrowattIP>/sta_config.html
- The default username/password is admin/admin
Change address from server.growatt.com to the ip address of the raspberry pi

Press “Apply”
At this point the traffic to server.growatt.com is flowing through your raspberry pi

Step 4. Prepare the Raspberry Pi for capturing

Processing of the Growatt Wifi Module data requires that bc, tcpdump and editcap (part of wireshark) are installed.

Update repositories: sudo apt update
Install programs: sudo apt install bc wireshark tcpdump curl host
create directory structure:
- sudo mkdir /home/pvoutput
- sudo mkdir /home/pvoutput/logs
- sudo mkdir /home/pvoutput/input
- sudo mkdir /home/pvoutput/processed
- sudo mkdir /home/pvoutput/scripts
- sudo mkdir /home/pvoutput/tmp

Step 5. Obtain PVoutput and Growatt Codes

To install the script we need some data

Growatt Inverter serial number.
- The serial number can be found on the side of the inverter, or under your Growatt Account at http://server.growattc.om
- Make sure to use the serial number of your inverter, not the serial from wifi module. My (correct) serial number formats like with SZxxxxxx. The Wifi module serial formats like YUxxxxxx (Don’t user that one)

PVoutput system ID and API
- I assume you already created a account and added your system to PVoutput.
- Go to: https://pvoutput.org/account.jsp
- Enable API access and copy your SystemID and API code;

Step 6. Install the capture/upload script

Now it’s time to download the script.

process_growatt_pvoutput Download

Unpack the script and edit the following lines:

PVOUTPUTKEY="your PVOutput API key" PVOUTPUTSID="your PVOutput System ID" GROWATTSERIAL="your Growatt inverter serial number"

For my own installation I also edited the following lines:
PVOUTPUTVPV1=YES PVOUTPUTVPV2=YES PVCUMFLAG=1 GROWATTMODULEVER=3.0.0.0

The default values were:
PVOUTPUTVPV1=NO PVOUTPUTVPV2=NO PVCUMFLAG=1 GROWATTMODULEVER=1.0.0.0

You might want to change values for your own situation

If you decided to connect the Raspberry Pi on Wifi rather than physical cable (eth0) change the line: INTERFACE=eth0 to INTERFACE=wlan0 as well. To make sure you have the right interface excute ifconfig and make sure you put the same interface name in the script as the one the ipadress is bind to.

Rasbian/Debian “Bullseye”
edit the line 188 where it says “/usr/sbin/tcpdump” and make it “/usr/bin/tcpdump“

Now place the process_growatt_pvoutput.sh into /home/pvoutput/scripts directory and make the script able to run by executing:
sudo chmod a+x /home/pvoutput/scripts/process_growatt_pvoutput.sh

Now the script is ready and it’s time for scheduling it every 5 minutes to make sure that the data capture files are processed and send to the pvoutput.org server.
sudo crontab -e

Add the following lines to the crontab
*/5 * * * * /home/pvoutput/scripts/process_growatt_pvoutput.sh >/dev/null 2>&1

Now you are all set and data is flowing to PVoutput. Congratulations!

If something goes wrong you can check the log with cat /home/pvoutput/logs/growatt_process.log

Step 7. Cleanup routines (optional)

The capturing and processing of the Growatt Wifi Module data generates quite a number of data files and log information. Not all of this information should be kept for a long time. To prevent the filesystem running full the following steps are recommended to perform.

sudo crontab -e
Add the following lines to the crontab
# Remove - correctly processed files - older then 7 days 30 0 * * * find /home/pvoutput/processed -name "growatt*ok" -type f -mtime +7 -exec rm {} \; # Remove - original capture files, which split into new files - older then 30 days 30 0 * * * find /home/pvoutput/processed -name "growatt*split" -type f -mtime +30 -exec rm {} \; # Remove - files not correctly uploaded - older then 30 days 30 0 * * * find /home/pvoutput/processed -name "growatt*badupload" -type f -mtime +30 -exec rm {} \; # Remove - original capture files that were to small - older then 30 days 30 0 * * * find /home/pvoutput/processed -name "growatt*size" -type f -mtime +30 -exec rm {} \;

Now add some logrotation
sudo nano /etc/logrotate.d/growatt_pvoutput

Add the following code:
/home/pvoutput/logs/growatt_process.log { daily rotate 7 compress }

Step 8. Daily Reboot (optional)

For stability I have put in a daily reboot every night at 1:10 am since the sun is not shining anyway at that time 😉
sudo crontab -e

Add:
#Reboot 10 1 * * * sudo reboot

Step 9. Update update update (recommended)

For security and stability reasons you need to update your Pi right after installation and preferably periodically;sudo apt update sudo apt upgrade
sudo reboot

Step 10. Backup your installation (optional)

To make sure your good work for all these previous steps are safe I recommend making a backup of your installation using Win32DiskImager

I also put in an automatic image backup to my Synology NAS. To do so:

Create a share on your Synology NAS (or windows server)
- e.g. pibackup
Create a user with access to the share
- e.g. pibackup
Install the require packages for CIFS access to your Raspberry Pi
- sudo apt install samba-common smbclient samba-common-bin smbclient cifs-utils
Create a directory to wich the CIFS share can be mounted
- sudo mkdir -vp /cifs/pibackup
Create the backup script
- sudo nano /home/pi/sdbackup.sh
- Put the following content in the script
  sudo mount -t cifs //<your synology or other cifsserver>/<sharename> /cifs/pibackup -o user="<user for cifs access>",pass="<password>",sec=ntlmv2,vers=3.0 sudo dd bs=4k if=/dev/mmcblk0 | gzip > /cifs/pibackup/rpi-pvoutput.fullbackup.$(date +%Y-%m-%d).img.gz sudo umount /cifs/pibackup
Make the script executable
- sudo chmod a+x /home/pi/sdbackup.sh
Schedule the script to run weekly (make sure it is started after your daily reboot (Step 8)
- sudo crontab -e
- Add the following
  30 1 * * 6 /home/pi/sdbackup.sh

29 Replies to “Upload Growatt Data to PVoutput with Raspberry Pi”

pinzvidz says:

28 January 2020 at 05:54

Thank you for this. Followed carefully and the data is going to Growatt as intended via the Pi. However, not getting any data through to PVoutput. Double-checked everything but to no avail…

Reply
- Edwin says:
  
  28 January 2020 at 08:59
  
  Please make sure you picked the right APi key. The read-only API will not work.
  
  Reply
  - pinzvidz says:
    
    28 January 2020 at 14:52
    
    Definitely using API, did not make a read-only key. I might start again just in case I missed something. Thanks again.
    
    Reply
    - Jrztn says:
      
      30 January 2020 at 11:42
      
      Hi pinzvidz,
      
      I have the same “issue”. Connected with the Growat Server and sending data perfectly.
      Not able to see the data through PVoutput.
      
      Did you already succeed?
      
      Reply
      - Edwin says:
        
        30 January 2020 at 14:11
        
        I Checked and notices 1 difference with my own installation;
        I had the following line added to the /etc/rc.local
        sudo bash /home/pvoutput/scripts/process_growatt_pvoutput.sh
        
        Updated the instructions
Jrztn says:

31 January 2020 at 12:21

Hi Edwin,
The rest of the instructions are great! thanks 🙂
I’ve added the line in rc.local but still nothing on PVoutput.
I’ll search for some logging and try the instructions again this weekend.

Reply
- Edwin says:
  
  31 January 2020 at 16:05
  
  For my own installation I also edited the following lines:
  PVOUTPUTVPV1=YES
  PVOUTPUTVPV2=YES
  PVCUMFLAG=1
  GROWATTMODULEVER=3.0.0.0
  
  The default values were:
  PVOUTPUTVPV1=NO
  PVOUTPUTVPV2=NO
  PVCUMFLAG=1
  GROWATTMODULEVER=1.0.0.0
  
  You might want to change values for your own situation
  
  To test the script you can run it from the shell promt by running
  sudo /home/pvoutput/scripts/process_growatt_pvoutput.sh
  
  Then watch the outpuut
  
  Reply
Edwin says:

2 February 2020 at 10:08

Make sure to use the serial number of your inverter, not the serial from wifi module. My (correct) serial number formats like with SZxxxxxx. The Wifi module serial formats like YUxxxxxx (Don’t user that one)

Reply
- Jrztn says:
  
  3 February 2020 at 19:38
  
  Everthing I did is correct. But still no data in pvoutput, i think my wifi dongle is different with some new , maybe encrypted, software. My serialnumber of the wifi dongle starts with JPCxxxxxx. In de growat serial i stored my growat serial starts with: QJBxxxxxx (Growatt 2000).
  My wifi firmware version is 1.7.6.6
  I’ll look further on the internet for some solution of the 2nd? generation wifi modules
  
  Reply
Jrztn says:

3 February 2020 at 19:51

Hi Edwin,

I’ve found a new way to store all my data. I use the P1 meter to read out my energy meter and store it in Home Assistant. As well as there is a growatt sensor api that reads data from the server.growatt.com.
Thanks for supporting 🙂

Reply
- Edwin says:
  
  4 February 2020 at 20:15
  
  I also read out my data from the P1 meter in Domoticz. But i wanted to publish it in the PVoutput Community
  
  Reply
Jan Siderius says:

6 March 2020 at 10:59

Ik heb de instructies gevolgd. Als ik het script wil opslaan krijg een foutmelding. Dir scripts is aangemaakt door ROOT, ik meld mij aan op de raspberry met PI. Hier zit het probleem. Hoe kan ik het script in de juiste directory plaatsen. ??

Reply
- Edwin says:
  
  7 March 2020 at 09:38
  
  Je kan het script eerst in de homedirectory plaatsen van de user “pi” en dan verplaatsen met:
  
  “sudo cp /home/pi/process_growatt_pvoutput.sh /home/pvoutput/scripts/process_growatt_pvoutput.sh”
  
  Als je voor commando’s “sudo” plaatst kan je ze uitvoeren als root.
  
  Reply
Jan Siderius says:

8 March 2020 at 17:04

Het werkt nu. Ik heb nog een raspberry met DSMR READER https://github.com/dennissiemensma/dsmr-reader. Ik stuur mijn slimme meter standen hiermee naar PVoutput. Kan ik DSMR reader zonder problemen op de raspberry installeren waar de Upload Growatt Data op staat ??

Reply
Andi says:

31 March 2020 at 08:50

Hi,

thanks for the good work, providing the scripts!!!

I just added the Wifi Module and set a static IP for it within my FritzBox Router. Connection to server.growatt.com is working, but I cannot login to the dongle via http://IP-ADRESS/sta_config.html in order to change the server address. I alway get an ERR_CONNECTION_REFUSED.

Maybe they changed login behaviour or am I missing something? Any help appreciated.

Thanks, Andi.

Reply
- semi says:
  
  12 April 2020 at 13:28
  
  Hi Andy,
  
  I got the same issue, i figured out that when the stick is connected with a wifi network you aren’t able to access the webinterface.
  
  So if you want to change settings you have to press the button on the wifi stick, make a direct connection and make your changes en connect back to your normal wifi network.
  
  Very inconvenient.
  
  Reply
  - Andi says:
    
    23 April 2020 at 09:24
    
    Thanks, I will try this.
    
    Reply
  - Andi says:
    
    28 April 2020 at 10:15
    
    Hi Semi,
    
    it seems, with the newer Stick Versions you are not able to capture the communication as it seems to be encrypted. I was able to changed the server on the stick, but after each communication it automatically switched back to server.growatt.com. I than created an WiFi AccessPoint with a Raspberry Pi and was able to sniffer the communication between the Stick and growatt. Here is the capture:
    
    11:07:58.937831 IP 192.168.1.240.3813 > 47.91.67.66.5279: tcp 585
    0x0000: 4500 0271 017a 0000 fe06 83d7 c0a8 01f0 E..q.z……….
    0x0010: 2f5b 4342 0ee5 149f 0001 5c3e 7feb 6a8d /[CB……\>..j.
    0x0020: 5018 1401 c0fc 0000 001c 0006 0241 0104 P…………A..
    0x0030: 0d22 2c43 5840 407e 332d 7761 7474 4772 .”,CX@@~3-wattGr
    0x0040: 6f77 6174 7447 726f 7761 7474 4772 2139 owattGrowattGr!9
    0x0050: 2231 4d73 475f 4659 7474 4772 6f77 6174 “1MsG_FYttGrowat
    0x0060: 7447 726f 7761 7474 4772 6f77 7570 684c tGrowattGrowuphL
    0x0070: 7555 7561 7474 3b72 6f77 6174 7447 726f uUuatt;rowattGro
    0x0080: 7761 7474 4772 6f77 6174 7447 726f 7723 wattGrowattGrow#
    0x0090: 747d ed72 707f ea74 7447 726f 7761 7474 t}.rp..ttGrowatt
    0x00a0: 4772 6f77 6174 7447 726f 7761 7474 4772 GrowattGrowattGr
    0x00b0: 6f77 6174 7447 726f 7761 7474 4772 6f77 owattGrowattGrow
    0x00c0: 6167 f34e 346f 7d61 7474 4772 6f77 6174 ag.N4o}attGrowat
    0x00d0: 7447 726f 7761 7474 4772 6f77 6174 7447 tGrowattGrowattG
    0x00e0: 726f 7761 7474 4772 4077 7526 7247 726f rowattGr@wu&rGro
    0x00f0: 7761 7474 4772 6f77 6174 7447 726f 7761 wattGrowattGrowa
    0x0100: 7474 4772 6f77 6174 7447 726f 7761 7474 ttGrowattGrowatt
    0x0110: 4772 6f77 6174 7447 726f 7761 7474 GrowattGrowatt
    
    Nothing in Plain Text except “Growatt”…
    
    Kind regards, Andi.
    
    Reply
    - David says:
      
      11 July 2020 at 04:47
      
      Hi All
      I also have tried to alter the server to the RPi but it won’t stay on and goes back to auto. The Shine WiFi dongle is new (2 weeks old). It also has a different GUI than that in Step 3.
      Any other ideas?
      David
      
      Reply
semi says:

12 April 2020 at 13:27

Hi,

I got everything setup, data is going to the pi and growatt servers.

I see the files in the processed folder: growatt_20200409_06:32.cap.split

Got everything setup with pvoutput (api etc) but no luck getting data uploaded.

I got the black Wifi stick (Growatt Shine WIFI-s) and a growatt 1500s converter.

Can you help me any further ?

Reply
Aart says:

20 April 2020 at 15:46

Hi Edwin,
Zaterdag omvormer vervangen door een Growatt 2000s met Shine Wifi-s dongle, en gelijk maar weer een Rasp omgebouwd met jou script. (Hulde daarvoor)
De serienummers hebben een nieuw format DLExxxxxCF de omvormer en de dongle JPCxxxxxCE heeft firmware 1.7.6.6.
Met de tcp dump pakketten kan ik in wireshark niet achterhalen waar het omvormer serial staat .(ook niet met HEX –> Text) In je blog zag ik eenzelfde opmerking staan , zonder een antwoord.
Ik heb met offset van 6 en van 0 geprobeerd maar dat blijft gokken…
De log blijft gelijk om de 5 minuten:
Capture file < 300 chars, moved to processed dir. : growatt_20200420_08:50.cap.smallsize
Capture file split, moved to processed dir. : growatt_20200420_08:51.cap.split
Invalid file removed : growattsize_00000_20200420085123
No input files to be processed. Processing ended
Enig idee waar ik naar moet kijken? hoe ziet een 1.0.0.0 of een 3.0.0.0 dump eruit?
Groet,

Reply
Richard says:

27 April 2020 at 19:11

Just followed your manual on my existing RPi. I did still have to install curl, along with the other tools.

I’m using the ShineLAN adapter (ethernet), which needs version 3.0.0.0 enabled inthe config. Other than that everything was exactly as the manual stated. So a warm thanks goes your way, Edwin. I am curious though what the VPV1 and VPV2 options are which you had enabled, but which should be disabled by default.

Cheers,
Richard (Maarssen, NL)

Reply
Darryl says:

8 June 2020 at 07:58

Great article and all operational except for uploads to PVOutput.

Are you able to confirm changes required for the following configuration
Growatt TL3 series inverter on 3 Phase with provisions for RST voltage terminology
Shine Wifi-S data logger (not the previous serial versions)
Support for PVOutput https (TCP port 443) https://pvoutput.org/service/r2/addstatus.jsp

3 Phase Voltage data Vac(R)(V), VacS(V), VacS(V) with conversion to Line voltage (see online conversions but essentially divide by √3, aka 1.732)
3 Phase generation data Ppv1(W) and Ppv2(W) and total in Ppv(W)

Currently nothing is being written to the input file as I believe it relates to the changes in raw data between single and 3 phase column headers and potentially the WiFi-S firmware and confirming the correct offset.

Can provide 3 Phase example data dump from growatt portals for reference if required.

Reply
maurice says:

11 June 2020 at 04:21

Hallo Edwin,

Vraag, zou dit ook kunnen werken voor 2 Growatt omvormers (1 via Ethernet en 1 via Wifi)
Zo ja wat moet ik dan allemaal aanpassen?

Met vriendelijke groet,
Maurice Brouns

Reply
- Edwin says:
  
  11 June 2020 at 07:28
  
  Hoi Maurice,
  
  Zoals ik in mijn artikel al zei heb ik het script niet zelf geschreven maar is de originele pagina niet meer online. Ik zou een Pi per omvormer installeren denk ik. Het lijkt me met dit script niet mogelijk om dit voor 2 omvormers tegelijk te realiseren.
  
  Gr,
  Edwin
  
  Reply
Jan van Duin says:

29 August 2020 at 08:04

Edwin, thanks for your work. I was already using this script several years and wanted to replace the capturing to another Raspberry Pi (which is also hosting my Domoticz and MQTT server). I couldn’t find the original page but luckily stumbled onto yours. Thanks to that I have it working like it should.

A few notes:
– In my configuration I can only login to my ShineLink bridge. The network setting to send the data to my Pi is done there, in Network Settings.
– I altered the bash script so it sends the Growatt data also to my local SQL database.
– After changing these settings I had to do a cold reboot before the new settings became active (power off/on).
– Somehow the data send by my Growatt inverter is scrambled. I found a python script made by Joris van der Els (remove_growatt_mask.py) which did the trick for me. (Objective: remove “growatt” bitmask from growatt message) Don’t ask me how, I have no knowledge about these things, but it worked. I can provide the script to anyone interested by mail: janvanduin.

Reply
- Wynand Marius de Wit says:
  
  16 September 2020 at 05:52
  
  Hi Jan,
  
  Thank you for the information. Will it be possible to provide the script please?
  
  I have the data, but am stuck with trying to remove the Growatt mask.
  
  Thank you,
  Wynand de Wit
  
  Reply
- Wynand de Wit says:
  
  17 September 2020 at 01:03
  
  Hi Jan,
  
  Will you be able to email me the script please?
  
  Regards,
  Wynand de Wit
  
  Reply
Vivian says:

30 December 2021 at 10:16

We are a Chinese based manufacturer focuses on the development and production of Firefighter safety switch (i.e., Rapid shutdown).
We are looking for partners in Europe,if you are interested, please contact me for further details.

Reply