Please see below
Subject: | StratML Import file feature through FTP is ready |
---|---|
Date: | Fri, 6 Jan 2023 23:46:24 +0530 |
From: | Sudarshana <sudar...@epicomm.net> |
To: | Naval Sarda <nsa...@epicomm.net> |
Hi Owen
We have deployed application of Import File Feature through the FTP.
If you upload files in /home/startml/Uploaded_xml_Files/ location on Server that files will get indexed after processing.
We are assuming that those files are getting uploaded
externally on https://stratml.us/docs/
this location.
Please see below
Subject: | Re: Fwd: StratML Import file feature through FTP is ready |
---|---|
Date: | Mon, 9 Jan 2023 23:23:57 +0530 |
From: | Sudarshana <sudar...@epicomm.net> |
To: | Naval Sarda <nsa...@epicomm.net>, Balasaheb Pandarkar <balas...@epicomm.net>, kom...@epicomm.net, Jitendra Shende <jite...@epicomm.net> |
---|
Owen,
We have changed startml folder name to stratml.
In another email we are sending you credentials for FTP, that will open one folder where you have to upload files.(/home/stratml/Uploaded_xml_Files)
Our program will run every 30 minute and picks up files from
this folder. Then another program will do processing on that
files and if files are perfect for indexing then add those files
in indexing folder. Then those files will get deleted from the
FTP folder.
After 30 minute when we search on we application that files
should be visible in search query.
If names are identical then file will get overwritten and if filename is different then new file gets added.
To explain this import feature I will explain you with one example.
I have taken one file 1MMMM.xml, renamed it with Test10.xml and
imported with this feature.
When I searched by Stakeholder name "Hal Burrows", I got 2 records in result. When I click on view link of first record it redirect to "https://stratml.us/docs/1MMMM.xml" URL. As this file is present on https://stratml.us , that opens successfully. When I click on second link, that will redirect to "https://stratml.us/docs/Test10.xml" URL. As this file is not present on https://stratml.us, it cant open and gives error of "Resource Not found".
For this we have to upload that file on https://stratml.us also. As this location is external to us and located out of the server where the program is running.
So, while uploading file on FTP you have to upload it on https://stratml.us/docs location
also. Please refer below images.
See below
-------- Forwarded Message --------
Subject: Re: StratML Import file feature through FTP is ready
Date: Fri, 6 Jan 2023 22:34:10 +0000 (UTC) From: Owen Ambur <owen....@verizon.net> Reply-To: Owen Ambur <owen....@verizon.net> To: Naval Sarda <nsa...@epicomm.net> CC: aboutthe...@googlegroups.com <aboutthe...@googlegroups.com>
Hi Owen,
We will check on the FTP issue you have mentioned.
Third party upload will be phase 2. That will involve approval process as we will need to figure out giving option to update existing listing as well through web interface. That time we can added third party links.
Naval
Hi Owen,
We had discussed FTP feature uploading stratml file. Now you have
suggested that you want the URLs as well.
We are suggesting a feature by which the FTP file you will be
uploading will have only the URLs and program will download the
content of the stratml file from the web and either insert new
record in database or update if that URL is already present. This
implementation will take additional 16 hours.
This will solve most of the issues like over riding and even open
the gateway to implement auto refreshing the database in future as
we will have URLs which we can use to download and reindex the
database on periodic basis. Auto refreshing is separate task and
not included in current estimates.
Naval
Please see below
Subject: | Re: Fwd: StratML Import file feature through FTP is ready |
---|---|
Date: | Tue, 10 Jan 2023 20:59:10 +0530 |
From: | Sudarshana <sudar...@epicomm.net> |
To: | Naval Sarda <nsa...@epicomm.net>, Balasaheb Pandarkar <balas...@epicomm.net>, kom...@epicomm.net, Jitendra Shende <jite...@epicomm.net> |
Owen,
Please use port number 21 to upload files. Permission denied issue is now resolved.
Currently we have implemented basic import
functionality with FTP, considering that this will
be used by you for your internal purpose . As we are
doing this through FTP, to update the existing files
we have to use overwrite functionality if name of
the files are same. If we assign random number after
each file, then it won't update existing record and
there will be lot of duplicates.
Along with file If you want to assign the URL for
that file where it should open on View Hyperlink, we
can provide one input control on web UI to provide
that URL. We will then save that URL along with the
uploaded files. It will take another 16 hours of
efforts along with initial 16 hours of web UI to
upload the files. That web UI will be visible after
login only. This login will be only for you. Not for
public. You can take call whether to implement in
such way or not. Direct FTP won't map the file with
the URL, hence web UI is needed.
Regarding public submissions of stratml files, we
can assign random numbers to the files they are
uploading and save the unique filename so generated
in database, so that they can update the records
they have uploaded using web UI. This is
essentially next phase and is not part of scope of
current phase.
-Sudarshana
Hi Owen,
Query service currently imports the data on submission once. But
if people are updating the StratML file on their server and are
not planning to resubmit again, it is not going to update in our
database as it is one time import funcationality.
This is where refreshing the query search database in needed.
This is similar concept where google indexes our website
periodically by crawling occasionaly. So whatever changes we make
to our website, may get indexed in google after few days and not
immediately. We have not estimated for auto refreshing the
database functionality.
Naval
Hi Owen,
See below email where you had approved the flow of URL import
feature. Both emails in the email chain below back to back where I
have mentioned the flow and you had replied approving the flow.
Naval
Hi Owen,
We have fixed permission denied issue. Please test again.
Naval
Hi Owen,
I have shared one email regarding FTP connection with two attachments. If it gets blocked and does not reach you, let me know as one of the files has .xml extension.
Naval
Yes, it will import all.
Naval