Python client

[A Kruger]

This is relevant to Linux users, and anyone interested in writing a
Psiphon 3 client.

I have started writing a command line Psiphon 3 client. You can find
it here: https://bitbucket.org/psiphon/psiphon-circumvention-system/src/4d1161698481/pyclient.
This is not yet ready to be released. But it should be usable.

The purpose of this client is:

1. to provide basic support to Linux users
2. to provide a clear demonstration of the Psiphon 3 API for anyone
who would like to write a new Psiphon 3 client

In order to run the Python client you will need to:

1. download the python files from bitbucket
2. construct a valid psi_client.dat file (see below)
3. run the command "python psi_client.py" from a command line.
4. launch a browser and configure it to use the SOCKS proxy at
127.0.0.1:1080

The psi_client.dat file must contain a JSON object containing the
following:
"propagation_channel_id": a string, like "FFFFFFFFFFFFFFFF"
"sponsor_id": a string, like "FFFFFFFFFFFFFFFF"
"servers": an array of strings. Each one is a server entry like those
found in the Windows Registry at HKCU\Software\Psiphon3 (Windows
client). You can open the Servers value and copy the first line of
data to use as the first element of this array.

All feedback is welcome.

Thanks.

[Amin Ab]

I'm using linux, (debian) and I didn't get that part that how to construct a valid psi_client.dat?! I don't even know what it is! :D

[Adam Kruger]

On Sat, Dec 1, 2012 at 7:57 AM, Amin Ab <medi...@gmail.com> wrote:

I'm using linux, (debian) and I didn't get that part that how to construct a valid psi_client.dat?! I don't even know what it is! :D

Hi.  The psi_client.dat file should be a regular text file with the following contents:

{"propagation_channel_id": "FFFFFFFFFFFFFFFF", "sponsor_id": "FFFFFFFFFFFFFFFF", "servers": ["fill me in"]}

You need to replace "fill me in" with the first line of your Servers registry value from a computer where Psiphon 3 has run.

[Mostapha Ramadan]

Dears thanks for your cooperation , 

i know we can get the servers from windows through this "HKCU\Software\Psiphon3" but what if i dont have windows on my pc! is there another way for that!

thanks in advance  

[Mostapha Ramadan]

it doesn't work for me , i've got the servers from windows client but i can't find what is the first line of the value ,, in the attachment you will find the out put of it ,

would you please help me ,, 

thanks in advance ,,,

[Adam Kruger]

Hi.  If you view your attachment in an editor with the "Word Wrap" option disabled then it will be easy to distinguish between lines.  Each line is a separate server entry.  If you're still having trouble, or if you are viewing with "Word Wrap" enabled, here's a hint: each line ends with "227D0A".

--
You received this message because you are subscribed to the Google Groups "psiphon3-developers" group.

To view this discussion on the web visit https://groups.google.com/d/msg/psiphon3-developers/-/AsDdDTLQQb8J.

To post to this group, send email to psiphon3-...@googlegroups.com.
To unsubscribe from this group, send email to psiphon3-develo...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/psiphon3-developers?hl=en.

[Mostapha Ramadan]

thanks alot for your kind reply , actually i did and find the whole line according on your response but infact it dosen't work! and i got error says " too many values to unpack " when i run the command " python psi_client.py " so i've attached the psi_client.dat may you please check it for me is it correct , 

your help very much appreciated ...

[Adam Kruger]

Hi.  If you are getting the "too many values to unpack" error, then I think that your source code is out of date.  Please pull down the latest version of the code.

I also noticed that the server entry that you are using is for an old server that no longer exists.  You can get the latest server entries from any Psiphon 3 download page.  The resource is called "server_list".  So if you visit a Psiphon 3 download page in your web browser (such as psiphon3.com), in the address bar replace "en.html" (or "fa.html") with "server_list" and you will download the latest server entries in a file.  The server entries are separated with "\n".

Please let us know if you have any more difficulties.

--

You received this message because you are subscribed to the Google Groups "psiphon3-developers" group.

To view this discussion on the web visit https://groups.google.com/d/msg/psiphon3-developers/-/S0BkRrVdlrEJ.

[Mohammads]

Thank you adam, but I can't start the Psiphon!!!
This is my "psi_client.dat":
{"propagation_channel_id": "FFFFFFFFFFFFFFFF", "sponsor_id": "FFFFFFFFFFFFFFFF", "servers": ["3130392e3232382e31392e31373120383038362038363762363162316664626130333934306439303831313537363963396434303333313030323865393933646462356438383765323732323233393266613064204d494943647a434341562b6741774942416749424144414e42676b71686b69473977304241515546414441414d4234584454457a4d4451794d7a49774e546b784e466f584454497a4d4451794d5449774e546b784e466f7741444343415341774451594a4b6f5a496876634e41514542425141446767454e414443434151674367674542414b314b73395670616245504635476f4638366a69336674745954796f4942626a6e4e31482b3462504a67304c366e7a796243535872736147625950304671364d7367726970487032473336595a76706254794a59355144364c526f46487269524d324d4f55533937787330324b67354a426e6135776d487447786e584f6337626f61756a5a5a524d53486652795832736c486d424d5234775847746244794d306870774737466c38475156484f724d44564b6a454f6765492b65384b6a2b676b6e536f68306443426c6367476c4c69616131436e633843754162587874475a353351375350424968717438756c7a4b546830766d656f7a4c664e706b4f414e415646476c2f6c4234394e636a565156693949336953764f76594579344933347630754c4b4436704c4445696c65475472777658674139434c564d72624f68682f3331686c693744776735784d3472507149713931396b4341514d774451594a4b6f5a496876634e415145464251414467674542414957714667676b6453772f52412b775570686a4b6c5052534832656431416775332f554b362f5362575a4a4c592f777a685378575843724f41726d55736f6c694766664d472b5634755367537a79386a4d7751625469426b594e4e4f4e486a4e58496b504a4142415167536b6c35576c766f797a5076414345554f5548342b48655870415454356f466c3278495a4f36537a666c777a47654f645978333866354d497a4165517a51304247546a374a6e4d4a336d636d6e6e4e4f356659785873786b6b6b6b3354466574373257695a2b686358766a76664d444e314b3178493067472b412f542f776548714e514743486b6379394e74726e586647666c39696453654b5a622f6f4d69717257525a7465456a67494a527a352b6a74516d7a3373484137584c6b6b4c6c50626572586770386a4c754c4c737641567a3043736561647a4d62476c626b5456306a71336b54454f66434a493d207b227765625365727665724365727469666963617465223a20224d494943647a434341562b6741774942416749424144414e42676b71686b69473977304241515546414441414d4234584454457a4d4451794d7a49774e546b784e466f584454497a4d4451794d5449774e546b784e466f7741444343415341774451594a4b6f5a496876634e41514542425141446767454e414443434151674367674542414b314b73395670616245504635476f4638366a69336674745954796f4942626a6e4e31482b3462504a67304c366e7a796243535872736147625950304671364d7367726970487032473336595a76706254794a59355144364c526f46487269524d324d4f55533937787330324b67354a426e6135776d487447786e584f6337626f61756a5a5a524d53486652795832736c486d424d5234775847746244794d306870774737466c38475156484f724d44564b6a454f6765492b65384b6a2b676b6e536f68306443426c6367476c4c69616131436e633843754162587874475a353351375350424968717438756c7a4b546830766d656f7a4c664e706b4f414e415646476c2f6c4234394e636a565156693949336953764f76594579344933347630754c4b4436704c4445696c65475472777658674139434c564d72624f68682f3331686c693744776735784d3472507149713931396b4341514d774451594a4b6f5a496876634e415145464251414467674542414957714667676b6453772f52412b775570686a4b6c5052534832656431416775332f554b362f5362575a4a4c592f777a685378575843724f41726d55736f6c694766664d472b5634755367537a79386a4d7751625469426b594e4e4f4e486a4e58496b504a4142415167536b6c35576c766f797a5076414345554f5548342b48655870415454356f466c3278495a4f36537a666c777a47654f645978333866354d497a4165517a51304247546a374a6e4d4a336d636d6e6e4e4f356659785873786b6b6b6b3354466574373257695a2b686358766a76664d444e314b3178493067472b412f542f776548714e514743486b6379394e74726e586647666c39696453654b5a622f6f4d69717257525a7465456a67494a527a352b6a74516d7a3373484137584c6b6b4c6c50626572586770386a4c754c4c737641567a3043736561647a4d62476c626b5456306a71336b54454f66434a493d222c20227373684f626675736361746564506f7274223a203738392c20227373684f6266757363617465644b6579223a202264393364393863396666333666623533636437653339363162646533306531613063353036396163323639323832303266393933646362356539643431656235222c20226361706162696c6974696573223a205b224f535348225d2c2022737368506f7274223a20302c2022737368486f73744b6579223a20224141414142334e7a614331796332454141414144415141424141414241514469496b56375475537143375566422f49524d4f535a334b6137412b72534631355878685141435a5532444e72714c78706e774d45792f6d59623432587632735258414a397453584a5365525569555045596931576f6a5a487659463558744a697557716167524a4453432b316d77435174744a747258346c65676f564d32675a4d69634b3043557666776e41416f6266612b5550352f4b6d38424a524253617a4a3575546571736f524a734f6c7432337858445133336e6f706f2b3769386e6f6e696b47374c4b56495a416d614b38532b39744575655357526648534c4c4451706d543946593847444f4658714f664353654344685935556679494632555a6a6b49704d62742f5232566f5a6f433458443458324f766f4967766468385078776f4b54787a676b30354c39784566515835684569514d656c6d7836466e4e674a794173796d77516f526239436a49714665646d3178222c2022776562536572766572536563726574223a202238363762363162316664626130333934306439303831313537363963396434303333313030323865393933646462356438383765323732323233393266613064222c2022737368557365726e616d65223a202270736970686f6e5f7373685f30386462336334353630393732333039222c2022776562536572766572506f7274223a202238303836222c2022697041646472657373223a20223130392e3232382e31392e313731222c202273736850617373776f7264223a202265313162613030356639346335356335393635643039323335303936616637386662336332343763633332333033363465333265333661393365316537623463227d"]}"servers": "servers": is one of complete lines of server key(Selected from server_list and not old)
but I don't know the correct value for "propagation_channel_id" & "sponsor_id"?
I copied "psi_client.dat" near the "psi_client.py" and this is the terminal output after executing python "python psi_client.py":

"too many values to unpack"

What should I do?
Help me please...
Thank you

[Adam Kruger]

Hi.  Please make sure that you have the latest version of the files from here.  The "too many values to unpack" error was fixed in psi_api.py:

line 37: self.ip_address, self.web_server_port, self.web_server_secret,

         self.web_server_certificate) = binascii.unhexlify(servers[0]).split(" ")[:4]

Your values for propagation_channel_id and sponsor_id are fine.

--

You received this message because you are subscribed to the Google Groups "psiphon3-developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to psiphon3-develo...@googlegroups.com.

To post to this group, send email to psiphon3-...@googlegroups.com.

Visit this group at http://groups.google.com/group/psiphon3-developers?hl=en.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[A Kruger]

The python client has now been merged into default.  You can find the latest source code here:

https://bitbucket.org/psiphon/psiphon-circumvention-system/src/b17d584afca81685ea33b672f66ee6ed5cbddc4a/pyclient?at=default

[Zhaozhou Li]

A piece of python code to convert server_list to psi_client.dat. May it be helpful for someone.

#!/usr/bin/python

import json

dat = {}

dat["propagation_channel_id"] = "FFFFFFFFFFFFFFFF"

dat["sponsor_id"] = "FFFFFFFFFFFFFFFF"

dat["servers"] = json.load(open('server_list'))['data'].split()

json.dump(dat, open('psi_client.dat', 'w'))

在 2013年1月1日星期二 UTC+8上午3:38:45，A Kruger写道：

To unsubscribe from this group, send email to psiphon3-developers+unsub...@googlegroups.com.

[jpe...@gmail.com]

Mod to psi_client.py for download and update servers

--------------------------------------------
import wget

# Delete 'server_list' if exists

if os.path.exists("server_list"):
    os.remove("server_list")

# Download 'server_list' and convert server_list to psi_client.dat

url ="https://s3.amazonaws.com//0ubz-2q11-gi9y/server_list"

wget.download(url)

dat = {}
dat["propagation_channel_id"] = "FFFFFFFFFFFFFFFF"
dat["sponsor_id"] = "FFFFFFFFFFFFFFFF"
dat["servers"] = json.load(open('server_list'))['data'].split()
json.dump(dat, open('psi_client.dat', 'w'))

...... psi_client.py

[Muhammad Ismail Soboute]

I really do thank your efforts in getting this python client ready. I hope everyone understands the magnitude of this work. I believe that at the moment this is the only means, for linux users behind firewalls and censoring programs, to experience net freedom.
I used to be a tor user on openSUSE and a regular fan for the last 3 years. But for some reason tor has stopped working on my machine since 2 month ago. It barely ever connects to the network and when it does, the speed is ridiculously low! Maybe the state have placed more aggressive traffic detection facilities to get their censorship ever more effective.
Anyway to get psipy client to work I made the said modifications to psi_client.py and I thought like sharing with you guys. Since I'm not a real python programmer, I'd rather not clone the repo and commit these changes there. But it would be nice if "Online updating of psi_client.dat" were added to the command line options. I did this, and here my psi_client.py:

#!/usr/bin/python
#
# Copyright (c) 2012, Psiphon Inc.
# All rights reserved.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
#


from psi_api import Psiphon3Server
from psi_ssh_connection import SSHConnection, OSSHConnection
import json
import os
import subprocess
import optparse
import wget


DATA_FILENAME = 'psi_client.dat'
CLIENT_VERSION = 1
CLIENT_PLATFORM = 'Python'
SOCKS_PORT = 1080
LOCAL_HOST_IP = '127.0.0.1'
GLOBAL_HOST_IP = '0.0.0.0'


class Data(object):

    def __init__(self, data):
        self.data = data

    @staticmethod
    def load():
        try:
            with open(DATA_FILENAME, 'r') as data_file:
                data = Data(json.loads(data_file.read()))
            # Validate
            data.servers()[0]
            data.propagation_channel_id()
            data.sponsor_id()
        except (IOError, ValueError, KeyError, IndexError, TypeError) as error:
            print '\nPlease obtain a valid %s file and try again.\n' % (DATA_FILENAME,)
            raise
        return data

    def save(self):
        with open(DATA_FILENAME+'.new', 'w') as data_file:
            data_file.write(json.dumps(self.data))
        os.rename(DATA_FILENAME+'.new', DATA_FILENAME)

    def servers(self):
        return self.data['servers']

    def propagation_channel_id(self):
        return self.data['propagation_channel_id']

    def sponsor_id(self):
        return self.data['sponsor_id']

    def move_first_server_entry_to_bottom(self):
        servers = self.servers()
        if len(servers) > 1:
            servers.append(servers.pop(0))
            return True
        else:
            return False


def do_handshake(server, data, relay):

    handshake_response = server.handshake(relay)
    # handshake might update the server list with newly discovered servers
    data.save()
    return handshake_response


def print_sponsor_message(handshake_response):
    home_pages = handshake_response['Homepage']
    if len(home_pages) > 0:
        print '\nPlease visit our sponsor\'s homepage%s:' % ('s' if len(home_pages) > 1 else '',)
    for home_page in home_pages:
        print home_page
    print ''


def make_ssh_connection(server, relay, bind_all):

    if bind_all:
        listen_address=GLOBAL_HOST_IP
    else:
        listen_address=LOCAL_HOST_IP

    if relay == 'OSSH':
        ssh_connection = OSSHConnection(server, SOCKS_PORT, str(listen_address))
    elif relay == 'SSH':
        ssh_connection = SSHConnection(server, SOCKS_PORT, str(listen_address))
    else:
        assert False

    ssh_connection.connect()
    return ssh_connection


def connect_to_server(data, relay, bind_all, test=False):

    assert relay in ['SSH', 'OSSH']

    server = Psiphon3Server(data.servers(), data.propagation_channel_id(), data.sponsor_id(), CLIENT_VERSION, CLIENT_PLATFORM)

    if server.relay_not_supported(relay):
        raise Exception('Server does not support %s' % relay)

    handshake_performed = False
    if not server.can_attempt_relay_before_handshake(relay):
        handshake_response = do_handshake(server, data, relay)
        handshake_performed = True

    ssh_connection = make_ssh_connection(server, relay, bind_all)
    ssh_connection.test_connection()

    server.set_socks_proxy(SOCKS_PORT)

    if not handshake_performed:
        try:
            handshake_response = do_handshake(server, data, relay)
            handshake_performed = True
        except Exception as e:
            print 'DEBUG: handshake request: ' + str(e)

    connected_performed = False
    if handshake_performed:
        print_sponsor_message(handshake_response)
        try:
            server.connected(relay)
            connected_performed = True
        except Exception as e:
            print 'DEBUG: connected request: ' + str(e)

    if test:
        print 'Testing connection to ip %s' % server.ip_address
        ssh_connection.disconnect_on_success(test_site=test)
    else:
        print 'Press Ctrl-C to terminate.'
        try:
            ssh_connection.wait_for_disconnect()
        except KeyboardInterrupt as e:
            if connected_performed:
                try:
                    server.disconnected(relay)
                except Exception as e:
                    print 'DEBUG: disconnected request: ' + str(e)
            ssh_connection.disconnect()


def _test_executable(path):
    if os.path.isfile(path):
        try:
            with open(os.devnull, 'w') as devnull:
                subprocess.call(path, stdout=devnull, stderr=devnull)
                return True
        except OSError:
            pass
    return False


def connect(bind_all, test=False):

    while True:

        data = Data.load()

        try:
            relay = 'SSH'
            # NOTE that this path is also hard-coded in psi_ssh_connection
            ossh_path = './ssh'
            if _test_executable(ossh_path):
                relay = 'OSSH'
            else:
                print '%s is not a valid executable. Using standard ssh.' % (ossh_path,)

            connect_to_server(data, relay, bind_all, test)
            break
        except Exception as error:
            print 'DEBUG: %s connection: %s' % (relay, str(error))
            if test:
                break
            if not data.move_first_server_entry_to_bottom():
                print 'DEBUG: could not reorder servers'
                break
            data.save()
            print 'Trying next server...'


def test_all_servers(bind_all=False):

    data = Data.load()
    for _ in data.servers():
        connect(bind_all, test=True)
        print 'DEBUG: moving server to bottom'
        if not data.move_first_server_entry_to_bottom():
            print "could not reorder servers"
            break
        data.save()

def update_data_file():

    if os.path.exists("server_list"):
        os.remove("server_list")

    # Download 'server_list' and convert server_list to psi_client.dat
    url ="https://s3.amazonaws.com//0ubz-2q11-gi9y/server_list"
    wget.download(url)
    dat = {}
    dat["propagation_channel_id"] = "FFFFFFFFFFFFFFFF"
    dat["sponsor_id"] = "FFFFFFFFFFFFFFFF"
    dat["servers"] = json.load(open('server_list'))['data'].split()

    json.dump(dat, open(DATA_FILENAME, 'w'))


if __name__ == "__main__":

    parser = optparse.OptionParser('usage: %prog [options]')
    parser.add_option("--expose", "-e", dest="expose",    
                        action="store_true", help="Expose SOCKS proxy to the network")
    parser.add_option("--update-data-file", "-u", dest="update_data_file",
                        action="store_true", help="Update data file (psi_client.dat) after downloading new server_list")
    parser.add_option("--test-servers", "-t", dest="test_servers",
                        action="store_true", help="Test all servers")
    (options, _) = parser.parse_args()
   
    if options.test_servers:
        test_all_servers()
    if options.update_data_file:
        update_data_file()
    elif options.expose:
        connect(True)
    else:
        connect(False)

[Adam Kruger]

Hi. Thanks for posting this. You might want to try our new cross-platform psiphon-tunnel-core (written in Go), which is used in our latest Windows releases, and will be used in all of our future platform releases.

--

You received this message because you are subscribed to the Google Groups "psiphon3-developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to psiphon3-develo...@googlegroups.com.

To post to this group, send email to psiphon3-...@googlegroups.com.

Visit this group at http://groups.google.com/group/psiphon3-developers.
For more options, visit https://groups.google.com/d/optout.

[Naman Tiwari]

I tried running the python client but cannot get any server to connect. I am getting Err110 Connection Timed Out. I've attached my psi_client.dat, which seems tome as the cause of the problem. Please help.

[Pulkit Chawla]

Hey everyone,

Use this repo for running psiphon on linux.

https://github.com/thispc/psiphon

It is Re-engineered and simple. psi_client.dat is now modified and made multifunctional. Option to update servers are provided in the same script.

Readme.md contains step by step instructions to set it up.

Reply in case of queries. 

[Muhammad Ismail Soboute]

Cheers pulkit

Muhammad Ismail Soboute
miso...@gmail.com
+989392625323 (cell)
+982122323553 (home)
48 Mirza Ali, Mellat Sq
N Majidiyeh, Resalat Ave.
Tehran, Iran, 1673834111

--
You received this message because you are subscribed to a topic in the Google Groups "psiphon3-developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/psiphon3-developers/cb8CW7Y98nI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to psiphon3-developers+unsub...@googlegroups.com.
To post to this group, send email to psiphon3-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/psiphon3-developers.

[Pulkit Chawla]

Hi. Check out https://github.com/thispc/psiphon . Contributors are Welcome . Please Give feedback

[Philip Collier]

It works well in Ubuntu 16.04, once the dependencies are installed. It takes much more time to establish a connection than the Windows and Android clients, but once it connects it is stable. Thanks for coding this!

[Subham Kumar Subham]

Hi

My college has time-based internet access and Psiphon Windows client bypasses it easily but when I tried  https://github.com/thispc/psiphon,  it doesn't work

When the internet is there it is able to create VPN but not otherwise.

[Reynaldo Gil-Pons]

thank you for porting it for linux. In my case I can access port 443 on servers, but not 53, it's blocked, is there any way to change the client accordingly? Original psiphon works in android as it uses 443 port and others...

[Dipen Patel]

how to choose own host server only ?

what setting required for choose only own host server ?

[Dipen Patel]

Hello..

i have started psiphond server ..

now i not getting idea how to get propagation_channel_id and sponsor_id value from server.

that value i required to set in client but i am not able to find it.

Thanks

[Dipen Patel]

yes i have read that but not getting idea.

u mean below 2 parametr value need to set Fix FFFFFFFFFFFFFFFF ?

"propagation_channel_id": a string, like "FFFFFFFFFFFFFFFF" 
"sponsor_id": a string, like "FFFFFFFFFFFFFFFF" 

Each one is a server entry like those 

found in the Windows Registry at HKCU\Software\Psiphon3   ?

i am using linux psiphon client and server so from where i get that.

Thanks 

--

You received this message because you are subscribed to a topic in the Google Groups "psiphon3-developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/psiphon3-developers/cb8CW7Y98nI/unsubscribe.

To unsubscribe from this group and all its topics, send an email to psiphon3-develo...@googlegroups.com.
To post to this group, send email to psiphon3-...@googlegroups.com.

[zeinab hajighorbany]

Hi Kruger, 

In our windows HKEY_CURRENT_USER\SOFTWARE\Psiphon3 there is no Servers  in the registry..