Re: [ECLK/results-tabulation] Integration points for pdf service

9 views
Skip to first unread message

Dinuka de Silva

unread,
Jun 28, 2020, 5:20:18 PM6/28/20
to Kosala Gangabadage, sherazad hamit, LSF-Elections
Hi Kosala,

I have just done the pdf service integration to tally sheets [1]. Could you please get this to the dev environment with the required configurations [2].

Regards,
Dinuka


On Wed, May 6, 2020 at 12:51 PM Dinuka de Silva <din...@opensource.lk> wrote:
Thanks. Let me check and get back to you

On Wed, 6 May 2020, 12:10 Kosala Gangabadage, <kos...@opensource.lk> wrote:

On Wed, May 6, 2020 at 11:44 AM Clement Fernando <clem...@opensource.lk> wrote:
Hi Dinuka,

1) Is this an internal service?
Yes. We are using it internally.
2) Is there any authentication or authorization flow included?
Nope. That’s why we use it internally.
3) Is this available in dev, stag and prod by now? If not, what is the timeline? And can you share the urls and required credentials?
Yes. Please obtain respective internal URLs from Kosala.
4) How is the reliability and load performance as of now. We have had bad experiences with the presidential election when we had pdfs generated at the backend as the load increased. If you have any statistics on response time, please share with me.
This we haven’t test.
5) Is there an advantage of having the templates[2] available inside pdf-service. And which are the template types allowed?
Mere advantage of having templates in the PDF services is, network speed. Yet again since we do internal calls, speed consideration could be negligible. But it still need to be tested and confirmed.

On May 5, 2020, at 18:11, Dinuka de Silva <din...@opensource.lk> wrote:

Hi Clement,

I have looked at the repository[1] roughly. But, for us to think of the integration points, can I get a small brief about pdf service.

Quick Questions
1) Is this an internal service?
2) Is there any authentication or authorization flow included?
3) Is this available in dev, stag and prod by now? If not, what is the timeline? And can you share the urls and required credentials?
4) How is the reliability and load performance as of now. We have had bad experiences with the presidential election when we had pdfs generated at the backend as the load increased. If you have any statistics on response time, please share with me.
5) Is there an advantage of having the templates[2] available inside pdf-service. And which are the template types allowed?

References

Regards,
Dinuka


Kosala Gangabadage

unread,
Jun 29, 2020, 12:06:07 AM6/29/20
to Dinuka de Silva, sherazad hamit, LSF-Elections
Hi Dinuka,

Configurations added to dev

Regards,

Kosala Gangabadage
Senior Software Engineer
Lanka Software Foundation 

Dinuka de Silva

unread,
Jun 29, 2020, 12:07:58 AM6/29/20
to Kosala Gangabadage, sherazad hamit, LSF-Elections
Noted and thanks kosala. 

Dinuka de Silva

unread,
Jun 29, 2020, 1:03:50 PM6/29/20
to Kosala Gangabadage, sherazad hamit, LSF-Elections
Hi Kosala,

Looks like the db hasn't been upgraded. As a part of pdf-service integration, a new column has been added to the tallySheetVersion table. Could you check please.

Note: db upgrade should run every time on any deployment. It doesn't have to be mentioned explicitly. 

Regards,
Dinuka

Kosala Gangabadage

unread,
Jun 30, 2020, 12:33:00 AM6/30/20
to Dinuka de Silva, sherazad hamit, LSF-Elections
Hi Dinuka,

DB upgrade has finished

Regards,

Kosala Gangabadage
Senior Software Engineer
Lanka Software Foundation 

Dinuka de Silva

unread,
Jun 30, 2020, 4:20:06 AM6/30/20
to Kosala Gangabadage, sherazad hamit, LSF-Elections
Hi Kosala,

There are changes in the swagger file. I think it hasn't been updated yet on apim.

Regards,
Dinuka

Kosala Gangabadage

unread,
Jun 30, 2020, 6:58:47 AM6/30/20
to Dinuka de Silva, sherazad hamit, LSF-Elections
Updated


Kosala Gangabadage
Senior Software Engineer
Lanka Software Foundation 

Dinuka de Silva

unread,
Jun 30, 2020, 7:10:39 AM6/30/20
to Kosala Gangabadage, sherazad hamit, LSF-Elections
Hi Kosala,

Need two things,

1) Show the environment variables on the tabulation api dev
2) Can you get me a log in the tabulation api dev.

Regards,
Dinuka

Kosala Gangabadage

unread,
Jun 30, 2020, 7:17:19 AM6/30/20
to Dinuka de Silva, sherazad hamit, LSF-Elections
Hi Dinuka,
Here are the ENVs
DEBUG = True

DATABASE_PLUGIN = "mysql+pymysql"
DATABASE_USERNAME = "lsf"
DATABASE_PASSWORD = "E!<4~>\\pd,EUPnP]"
DATABASE_HOST = "192.168.75.7"
DATABASE_PORT = "3306"
DATABASE_NAME = "election_prod"
PDF_SERVICE_URL = "http://pdf-service-1hcnse.pxe-dev-platformer-1552477983757-1pdna.svc:5000"

SQLALCHEMY_POOL_SIZE = 50

RESULT_DISSEMINATION_SYSTEM_URL = "http://result-dist-1crbnh.pxe-dev-platformer-1552477983757-1pdna.svc:8181"
RESULT_DISSEMINATION_SYSTEM_ELECTION_CODE = "foo"
RESULT_DISSEMINATION_SYSTEM_RESULT_TYPE_VOTE = "PRESIDENTIAL-FIRST"
RESULT_DISSEMINATION_SYSTEM_RESULT_TYPE_PREF = "PRESIDENTIAL-PREF"

PROD_ENV = False

Regards,

Kosala Gangabadage
Senior Software Engineer
Lanka Software Foundation 

Dinuka de Silva

unread,
Jun 30, 2020, 7:18:43 AM6/30/20
to Kosala Gangabadage, sherazad hamit, LSF-Elections
Noted. Give me the log also please.

Kosala Gangabadage

unread,
Jun 30, 2020, 11:04:23 AM6/30/20
to Dinuka de Silva, sherazad hamit, LSF-Elections
Hi Dinuka,

Please find attached log.

Kosala Gangabadage
Senior Software Engineer
Lanka Software Foundation 

Tabulations-dev-log.log

Dinuka de Silva

unread,
Jun 30, 2020, 11:23:56 AM6/30/20
to Kosala Gangabadage, Clement Fernando, Yujith Isura, sherazad hamit, LSF-Elections
Hi Kosala,

Could you send me pdf service logs too.


Actually it ends up with the following error. @Clement Fernando @Yujith Isura  Do you have any clue if you have experienced similar issues? 
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1354, in getresponse
    response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 276, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:


Regards,
Dinuka

Kosala Gangabadage

unread,
Jun 30, 2020, 11:27:01 AM6/30/20
to Dinuka de Silva, Clement Fernando, Yujith Isura, sherazad hamit, LSF-Elections
Hi Dinuka,

Herewith attached pdf service logs

Kosala Gangabadage
Senior Software Engineer
Lanka Software Foundation 

pdf-service-log-dev.log
Reply all
Reply to author
Forward
0 new messages