Technical Issues & Cost Estimate

[Owen Ambur]

Thanks for the cost estimate, Naval.  While I will be paying for results and not for hours, having the breakdown helps sharpen my understanding of the technical issues as you see them.

For example, first, with regard to the "need to remove some unwanted tags," there are no "unwanted tags" in StratML files.  They are all part of the standard.  However, if what you mean is that the database will not accept them, I'd like to know why.  

Also, why should it take as many as 36 hours to remove those tags?  Shouldn't it be possible to automate that process when they are imported into the database?

Moreover, since that will be an ongoing issue as more files are indexed, I'd want the process to be automated because it's not something I'd want to have to do manually each time another document or group of documents is indexed.

Finally, with respect to that issue, bear in mind that it means the files in the database will not be "valid" StratML instance documents.  Since the database is merely an metadata repository and not the authoritative source of the documents themselves, that's not necessarily a big issue -- unless users are led to believe they can export and save valid StratML files from the query service.  On the other hand, it should also be possible to reverse the process and reinsert the missing tags upon export from the database.

Second, I don't understand the need for loading a second database.  Why should that be necessary?  What is the purpose of the second database?  If it is indeed required, I'm not comfortable making a go/no-go decision without having a clear understanding of the cost and other implications.

Third, with respect to hosting, why should it take as long as 16 hours to figure out if my hosting service is adequate?  Since it would increase not just the cost but, more importantly, the administrative overhead, I'd prefer not to have to deal with another hosting service.

Fourth, I can see why it might take four hours or more to determine if BaseX is the best alternative to use as a database but I thought that determination had already been made, pending confirmation that it will run via my hosting service.

Fifth, while I can also understand why it might take 12 hours to design a stylesheet, I'd need to understand why the one that we already have is inadequate.  Beauty is in the eye of the beholder and presentation/styling can always be improved, but for purposes of the minimally viable query service, the existing stylesheet should be sufficient.  Enhancements could be considered in a follow-on project.

There are other issues I could raise with the itemized cost estimate but, again, the number of hours devoted to each task is not my concern.  Your projected cost is within the range of what I'm willing  to pay.  However, what I want to purchase is a query service that works as intended, and to the degree that I'm paying for software development, I'd like the code to be made readily available as open source, e.g., on GitHub.

I'm looking forward to resolving these issues so that we can move forward with a determination on how best to proceed from here.

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: Naval Sarda <nsa...@epicomm.net>
To: Owen Ambur <Owen....@verizon.net>
Sent: Tue, Oct 4, 2022 11:46 am
Subject: Re: Query & Results Screens

Hi Owen,

We have updated the screen as per your request. See attached.

Please see attached estimates as well. Total hours as per attached scope of work is 212 hours. Hourly rate is $18/hr. So total amount is $3816

Some of the task are already done and some are half done.

Let me know if you want to proceed with basic search feature.

Naval Sarda

EpiComm Technologies

-------- Forwarded Message --------

Subject:	Re: Query & Results Screens
Date:	Sat, 1 Oct 2022 18:01:56 +0000 (UTC)
From:	Owen Ambur <owen....@verizon.net>
Reply-To:	Owen Ambur <owen....@verizon.net>
To:	nsa...@epicomm.net <nsa...@epicomm.net>
CC:	aboutthe...@googlegroups.com <aboutthe...@googlegroups.com>

Lookin' good, Naval.

In the revised wire frames, the only editorial suggestion that I'd reiterate is to label the column with the hyperlink icon in the search results list "View" instead of "Action".  (If we decide to provide an Edit option later, we could add another column for that purpose but it might make more sense to enable that option after the document has been presented in view-only mode.

My thought has been that we should defer the advanced search capabilities till later but I defer to your expertise and judgement on whether they can be accmmodated without much extra effort.

With reference to item 3, no, unless it were necessary for system performance reasons, I would not want constrain the terms that users can enter for queries.

Regarding item 7, it has been my intention that clicking on the hyperlink in the search results listing will take users directly to the element within the context of the source of the plan/report wherever it resides on the Web.  However, as you suggest, it would also be good to enable reassembly and styling of the documents from their content stored in the query service database.

In the event the latter is the first or only option, it would be good to include a notice providing a link to the original XML source.  While the intent is that the XML sources will eventually be the authoritative sources, that is not the case with the documents in the existing StratML collection.  They have all been copied from other sources and the URLs are included in the <Source> element.

I hope building the query service will be as easy as it sounds like it might be.  I'm anxious to see how well it works.

What's the next step?

BTW, we just got back home from a week in the North Carolina mountains.  Thankfully, there was no particular damage here on Hilton Head Island from hurricane Ian.  We'll be here a week and then heading to my 50th college reunion in South Dakota for the week after that.

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: Naval Sarda <nsa...@epicomm.net>
To: Owen Ambur <Owen....@verizon.net>
Sent: Fri, Sep 30, 2022 10:28 am
Subject: Fwd: Query & Results Screens

Please see below

-------- Forwarded Message --------

Subject:	Re: Fwd: Query & Results Screens
Date:	Fri, 30 Sep 2022 19:46:54 +0530
From:	Sudarshana <sudar...@epicomm.net>
To:	Naval Sarda <nsa...@epicomm.net>
CC:	jitendras <jite...@epicomm.net>, Balasaheb <balas...@epicomm.net>

Please go through the inline comments in BLUE color and share your thoughts.

On 9/30/2022 9:51 AM, Naval Sarda wrote:

Naval, here are my thoughts on the updated wire frames:

1. The full text search field should be labeled "Full Text Search" or perhaps "Search Full Text". Changes made, please refer attached screenshot.
   
2. The specific element search fields should be labeled "Find Goals/Objectives" | "Find Stakeholders" | "Find Values" and the grayed out text in the search fields themselves should read: "Enter Goal/Objective Name" | "Enter Stakeholder Name" | "Enter Value Name". Changes made, please refer attached screenshot.
3. Users don't need to know that both the Name & Description elements are being queried, and including "Description" in the label implies that phrases and sentences can be entered as query terms.  While there should be nothing to prevent users from doing that if they choose, the idea is for them to enter a single word.  Changes made in screenshots, please refer attached screenshot. We can programatically prevent user from typing more than one word. If you want we can do that.
   
4. It would be good if the label above the search results listing were to include the word or phrase on which the query has been conducted, e.g., "Search Results for Education". We can do that.
5. In the basic Search Results listing, it seems to me that the only text that should be returned is the Description of the Goal/Objective along with the Name of the Organization compiling the plan, i.e., two columns plus the hyperlink icon, whose column should be labeled "View".  The Description column should be the widest. OK
6. The exceptions would be when the user is querying *only* the Stakeholder or the Value element, in which case the Description for that element should be displayed along with the Name of the Organization. OK
   
7. When only Stakeholders or Values are being queried, one complication is that the hyperlinks cannot point directly to them in the styled renditions of the documents because, unlike Goals & Objectives, Stakeholders and Values do not have GUIDs. Even though the Stakeholders or Values or any other field is queried, result displayed will be the one XML per row and we will create styled xml for that record to render on screen.  So that will not be the problematic.
   
8. In the case of Values, a work-around would be to link to the Values section, like this. OK.
9. Exactly how to deal with Stakeholders may be a bit more complex, since they may appear in three places -- under Organization, Goal, or Objective. Here's an example. Again same answer for this like point number 7
   
10. Another complication is what to display in the search results list if both Values and Stakeholders are queried at once but without querying Goals/Objectives as well.  In that event, it might make sense to display the Organization Name and the Mission statement (in the Description column). When more than one fields are queried then result will be the combination of both fields. Means those records will come in search result for which Value and Stakeholders both are equal to the quieried values. Same logic will be applied to all other fields.

Herewith we have attached screenshots for both search types. Basic search and the advanced search. In basic search we have made all those changes which you suggested in last email.

In advanced search as you asked in last email that you want to search for Goals/Objectives associated with particular Stakeholders, we tried to design screen accordingly. We thought that first user will start typing in stakeholders field, as he types he will get suggestions at the bottom. He can choose multiple stakeholder. Then when he start typing in goals/objectives, he will get suggestions with only stakeholders associated goals/objectives. Again user can choose multiple goals/objectives.

1. No doubt, there will be other complications and issues to be worked out as well, but I hope these thoughts are clear and meaningful in the meantime. You can share all your concerns, we will try to analyse that all.
   

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: Naval Sarda <nsa...@epicomm.net>
To: Owen Ambur <Owen....@verizon.net>
Sent: Thu, Sep 29, 2022 9:28 am
Subject: Fwd: Keyspaces &/or DynamoDB for StratML Query Services?

Please see below

-------- Forwarded Message --------

Subject:	Re: Fwd: Keyspaces &/or DynamoDB for StratML Query Services?
Date:	Thu, 29 Sep 2022 17:21:05 +0530
From:	Sudarshana <sudar...@epicomm.net>
To:	Naval Sarda <nsa...@epicomm.net>, Jitendra Shende <jite...@epicomm.net>, Balasaheb Pandarkar <balas...@epicomm.net>

Please see inline comments in blue color.

On 9/29/2022 4:24 AM, Naval Sarda wrote:

-------- Forwarded Message --------

Subject:	Re: Keyspaces &/or DynamoDB for StratML Query Services?
Date:	Wed, 28 Sep 2022 15:40:07 +0000 (UTC)
From:	Owen Ambur <owen....@verizon.net>
Reply-To:	Owen Ambur <owen....@verizon.net>
To:	nsa...@epicomm.net <nsa...@epicomm.net>
CC:	aboutthe...@googlegroups.com <aboutthe...@googlegroups.com>

Good progress, Naval.  Here are my initial thoughts on your updated screen shots/wire frames:

1. It seems to me that it would be good to combine each of the two search fields for Goal/Objective, Stakeholder, and Value into one so that the simple seach screen includes only three fields. Changes made in mockups. Please refer screenshot attached.
2. Ideally, it would be good if the user could decide to query only one or two or all three of those fields at once.  However, that may be more technical complexity than we should take on initially. We will manage that programatically.
3. Regarding the query results screen, I assume the pencil icon indicates an edit link, which is not feature I'd like to include in the basic query service.  However, at some point, I would not be adverse to providing such feature that opens the file in my XForm for editing and saving locally.  I don't want to get into the business of managing UIDs & passwords for authorizing edit access to the database. Removed pensil sign. Please refer screenshot attached.
4. With respect to the column headings, we'll need to give them further thought but the share of screen space devoted to the Description element should be increased.  That information will be the primary means by which users will determine which of the results they may wish to view in context of the documents in which they occur.Column size for 'Description' field has increased. Please refer screenshot attached.
   
5. Depending upon which of the elements the user has queried, it should not be necessary to show that element in the column heading.Ok. We will show only those columns which you are interested for.
   
6. I assume the green down arrow designates a download feature, which may be OK, but if there is an icon that indicates viewing/opening a hyperlink, that would be better.  Yes green is for download and we have added view/hyperlink button also. Please refer screenshot attached.
7. In that regard, one issue to be addressed is the fact that my sitemap listing references the plain XML text renditions whereas the styled renditions should be presented when clicking on links in query results listings. Ok
8. In some cases, PDF renditions are also available but they do no permit direct referencing of the queried elements, which is the primary purpose of the query service. Ok

BTW, Hilton Head Island is directly in the currently projected path of hurricane Ian.  While we're currently in the North Carolina mountains, we've been scheduled to head home on Friday but will probably need to delay our return and will be a bit unsettled for the next few days.

Please take care of yourself and family.

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: Naval Sarda <nsa...@epicomm.net>
To: Owen Ambur <owen....@verizon.net>
Cc: aboutthe...@googlegroups.com <aboutthe...@googlegroups.com>
Sent: Wed, Sep 28, 2022 10:57 am
Subject: Re: Keyspaces &/or DynamoDB for StratML Query Services?

Hi Owen,

We have designed another basic search screen. Please see attached.  We can further refined the same based on your instructions.

Thanks for providing the information below which tells us which all tags and section needs to be searched behind the scenes. It is really helpful.

We will get back to you about advance search screens.

Naval

On 27/09/22 8:09 pm, Owen Ambur wrote:

Naval, as summarized under the Query/Discovery goal in my technical development plan, the basic query service should enable searches of these elements of the schema:

1. Stakeholder Name & Description
2. Goal & Objective Name & Description
   
3. Value Name & Description

While those are the elements of the "basic" search capability, please note that it is not exactly "simple" because:

• The Stakeholder element occurs at three levels of the schema -- Organization, Goal & Objective.
  
• Since the only distinction between Goal and Objective is a matter of timing, it makes sense to query those elements together.
• Similarly, since the queried term may occur in either the Name or the Description, it might make sense to combine queries of those two elements together as well.  For the goal/objective query, that means searching four elements at once: 1) Goal Name, 2) Goal Description, 3) Objective Name, and 4) Objective Description.

More advanced query capabilities will enable such things as faceted searches, e.g., searching for goals/objectives associated with particular stakeholders.

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: Naval Sarda <nsa...@epicomm.net>
To: Owen Ambur <owen....@verizon.net>
Sent: Tue, Sep 27, 2022 12:28 am
Subject: Re: Keyspaces &/or DynamoDB for StratML Query Services?

Hi Owen,

For the search fields, we checked the plan again. But we were not able to figure out the search fields needed for basic and advance search. If you can just copy and paste the same in the email, it will be great.

Naval

On 24/09/22 8:24 pm, Owen Ambur wrote:

Naval, I always enjoy having the opportunity to virtually meet those with whom I have established potentially productive relationships and I'll be happy to join you in a televideo conference when convenient for both of us.

Please be aware, however, that my hearing is not great and I have a hard time understanding Pradeep on our televideo conferences.  Also, I wouldn't want progress to be held up until we can find a time that is mutually convenient for an synchronous meeting of the minds.

So from that perspective E-mail works better for me.  It also seems to me that your usage of it has enabled you to make more rapid progress than he has.

On the other hand, it would be great if we could all work together, with each of us adding the value accommodated by our own interests and expertise.  So I'd welcome the chance to have joint televideo conference with both of you.

I'll be especially interested to learn what it is about my plan that is still unclear.  To the degree that it needs to be further fleshed out, I'd rather do that than merely talk about it.  It contains mailto Comment links to facilitate such focused dialog.

BTW, you can check out what Pradeep has done in the past at https://stratml.us/#Ictect

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: Naval Sarda <nsa...@epicomm.net>
To: Owen Ambur <am...@verizon.net>; pradee...@ictect.com <pradee...@ictect.com>; jo...@vionta.net <jo...@vionta.net>
Sent: Fri, Sep 23, 2022 8:57 am
Subject: Re: Keyspaces &/or DynamoDB for StratML Query Services?

Hi Owen,

We need to get on call once you are back from your vacations so that we can understand your needs better.

Naval

On 23/09/22 1:41 am, Owen Ambur wrote:

Naval, Pradeep & Jorge, while I hope that we can build on experience with BaseX, I am also open to exploring other alternatives.

I don't understand the pricing implications of AWS Keyspaces.  However, I suspect that usage of a StratML query service might initially be pretty low, and if it scales up to become costly under those pricing conditions, that might indicate the potential for profitably enaging paying customers.

Keyspaces came to my attention via a posting on LinkedIn by Jason Hunter, who spent >17 years with MarkLogic and is now Principal Solution Architect for DynamoDB.  I don't understand its pricing implications for it either and would be hesitant to take on a "free" account without knowing what the costs might turn out to be.

Owen Ambur

https://www.linkedin.com/in/owenambur/

--
Thanks & Regards
Sudarshana

--
Thanks & Regards
Sudarshana

--
Thanks & Regards
Sudarshana

[Naval Sarda]

Hi Owen,

I have already mentioned in my email, that some tasks are already done like doing database comparison. You had emailed lot of options and we did evaluate those and we have emailed our findings.

And yes you will be paying for results only.

Regarding your existing hosting, it is shared hosting. As far as I know, you won't be able to install BaseX on shared hosting. You can contact their support and cross verify and also ask them what type of hosting they can provide so that we can install third party softwares particularly BaseX and tomcat (Java server).

We understand that StratML is a standard and there is no unwanted tags. But when we tried searching in BaseX, there was only particular attribute in the beginning of file which was not letting BaseX search to happen. Idea here will be to maintain two fields, only will original StratML XML. Other field will be modified XML used for searching.

Today (Wednesday) is holiday here in India.

https://en.wikipedia.org/wiki/Vijayadashami

We will update justification of hours in sheet itself. Hours are not going to reduce by any way. You have to understand BaseX is not something being used for traditional applications. So there is a bit of learning curve and we did spend time to understand and reach to the point that we are now comfortable to use BaseX for your application.

So if you see 16 hours in server management, it includes a bit of time to understand how it can be installed, etc.

In nutshell, you can wait for justification for each item for tomorrow. Or you can give goahead and we can continue detailing each item so that you understand each task better.

You can going to pay for results only.

Naval

[Owen Ambur]

Naval, see my chat with MochaHost copied below.

Looks like we're good to go.

I'm looking forward to seeing the application in action so that we can decide when to make it available on the AboutThem.info site.  However, there is no particular rush.  Enjoy your holiday.

Owen Ambur

https://www.linkedin.com/in/owenambur/

Mochahost at 10:45:57

Welcome to MochaHost 24/7 Live Chat. An agent will be with you momentarily!

Mochahost at 10:46:00

Hello! My name is Tray, and I have the pleasure to assist you today! Please, * Rate Your * experience at the end of this session.

Tray at 10:46:19

I'm here and ready to help with any question or issue you may have! Please let me know how may I be of assistance?

Tray at 10:46:22

Hello, Owen!

Tray at 10:46:26

Thank you for contacting with us!

Tray at 10:46:35

You're able to install it on a VPS server.

Tray at 10:46:47

We do have a Java VPS server that has a Tomcat on it.

Tray at 10:47:50

Here is the Java VPS package - https://mochahost.com/java.php

Owen at 10:48:02

I take that to mean your answer is yes BaseX can be installed and Tomcat is already available.

Tray at 10:48:11

Indeed.

Tray at 10:48:23

Since you have a root access to the VPS server , you would be able to install it.

Owen at 10:48:29

Thanks a bunch.

Tray at 10:48:29

There are no limitations on that environment.

Tray at 10:48:38

You're most welcome, Owen!

Tray at 10:48:38

Do you have any other requests, that I can assist you with?

Owen at 10:49:17

That's all. I just want to copy this chat so that I can pass it on to my developer.

[Naval Sarda]

Hi Owen,

Thanks for confirmation.

The Mochahost chat is suggesting to purchase Java VPS server and has pointed out link to subscribe for the same.

Once you have done that, please share access to your new Java VPS server (SSH access is needed to install BaseX on it).

Naval

[Owen Ambur]

More decisions to be made.  See the screen shots below.

I'm inclined to assume the application will work and I'll want to maintain it indefinitely.  So I should opt for the 3-year subscription to save a few bucks.  I'm also guessing that the lowest "heap" allowance should be enough, at least initially.  However, making that determination is beyond my level of technical expertise.

What do you advise?  How large a heap do you believe will be required?

The second screen shot shows their recommendation for heaps greater than 512MB.

Owen Ambur

https://www.linkedin.com/in/owenambur/

[Naval Sarda]

Hi Owen,

We have two apps BaseX and our webapp running together.  You need to go for VPS plans. Second screenshot provides link to VPS plans.

Buy only monthly plan. You can switch the same server to yearly plan later if everything works good.

Ideally we need 8GB Ram VPS server so that both BaseX and Java application can reside on same server. Anything lower than 8GB ram will be too tight I belive.

You can maintain indefinately.

Naval

[Owen Ambur]

Naval, as shown in the screen shots below, I started the process of acquiring a Java VPS plan.  However, I didn't complete the order because I'm not sure many cores or how much disk space may be required.  

Also, since the AboutThem.info domain is only one of my several sites hosted by Mocha, I hope updating my nameservers for that domain will not mess up access to the others.

Please advise.

I'm also getting a sense of the ongoing financial commitment to maintain hosting of the site.  It is considerably more than I am currently paying.  However, unless a better deal might be available elsewhere, I'm willing to pay it if the service works as I hope that it will and can be enhanced to provide addition capabilities in the months and years ahead.

Owen Ambur

https://www.linkedin.com/in/owenambur/

[Naval Sarda]

Hi Owen,

2 core is fine. Regarding nameserver, ideally when we are in development phase, we recommend to point a subdomain to the new application till it is complete.

For example app.AboutThem.info or qa.AboutThem.info or staging.AboutThem.info

Some people have marketing/info website on main domain and application on app.maindomain.com

Regarding pricing, we have been seeing these kind of prices for 8GB RAM server.

If you search for BaseX memory requirement in google, you might find people reporting BaseX Outofmemory error. So 8GB is recommended.

Naval

[Naval Sarda]

Hi Owen,

In addition to my previous response, Diskspace 100GB will be sufficient for our use.

Naval

[Owen Ambur]

Naval, as shown in the first screen shot below, the cost of the Java VPS service for the first month would be about $50.

As per the second screen shot, the lowest price for a three-year subscription would be about $2,000.

Since we'll be traveling next week, I think I'll wait until we get back to order a one-month plan to make sure we have plenty of time to make sure the application is working properly before I need to make a determination on extending it.

Owen Ambur

https://www.linkedin.com/in/owenambur/

[Naval Sarda]

Hi Owen,

Whenever you are back from vacation, please hire the VPS server. We are planning to deploy first APIs on coming Tuesday so that UI developer can start calling APIs on server.

Naval