How to read from an html file in a program in PreText?

[Barbara Ericson]

I am using BeautifulSoup in Python and was able to grab live data and process it in PreText in a Python program in an ebok.  However, more websites are not allowing live web scraping.  I saved the HTML page that I want to process to a file and want to read it in Python as a string and then create a soup object from it.  I tried using <datafile> as shown below, but it wouldn't build the book.  

<datafile xml:id="MOOC-wikip-id" label="MOOC-wikip-id" filename='MOOC-Wikipedia.html'>
       <pre source="bsoup/MOOC-Wikipedia.html"></pre>

</datafile>

See the following for what happened when I tried to build the ebook.  I think it is a problem with how it processes the files in assets.  I think it expects the file to not be html.  

I had the same problem with a database file and see that there is a different way to handle those types of files.  Is there a special way to handle html?

-------------------------------------------------------------------------------------------------------------------

(base) barbarer@Mac pretext % python -m pretext build web

Version 2.47.1 of pretext is available.  You are currently using version 2.44.0.

To upgrade, run `pretext upgrade`.

PreTeXt project found in `/Users/barbarer/Documents/GitHub/py4e-int`.

Building target web

XSL conversion of /Users/barbarer/Documents/GitHub/py4e-int/pretext/main.ptx by /Users/barbarer/.ptx/2.44.0/core/xsl/utilities/pretext-enhanced-source.xsl

successful application of /Users/barbarer/.ptx/2.44.0/core/xsl/utilities/pretext-enhanced-source.xsl

warning: Schema validation could not be completed: no validator was available (jing is not installed and lxml could not compile the schema). Continuing with build.

Generating any needed assets.

Finished generating assets.

Preparing to build into /Users/barbarer/Documents/GitHub/py4e-int/output/output/web.

discovering source file's directory name: /Users/barbarer/Documents/GitHub/py4e-int/pretext

verifying and expanding input directory: /Users/barbarer/Documents/GitHub/py4e-int/pretext/../generated-assets

input directory expanded to absolute path: /Users/barbarer/Documents/GitHub/py4e-int/generated-assets

verifying and expanding input directory: /Users/barbarer/Documents/GitHub/py4e-int/pretext/../assets

input directory expanded to absolute path: /Users/barbarer/Documents/GitHub/py4e-int/assets

discovering source file's directory name: /Users/barbarer/Documents/GitHub/py4e-int/pretext

XSL conversion of /Users/barbarer/Documents/GitHub/py4e-int/pretext/main.ptx by /Users/barbarer/.ptx/2.44.0/core/xsl/utilities/report-publisher-variables.xsl

successful application of /Users/barbarer/.ptx/2.44.0/core/xsl/utilities/report-publisher-variables.xsl

Downloading rs services file from https://runestone.academy/cdn/runestone/latest/webpack_static_imports.xml.

Runestone Services via online CDN query: version 8.2.2

Downloading Runestone Services, version 8.2.2

Extracting Runestone Services from archive file

Building custom css theme: theme-salem

converting /Users/barbarer/Documents/GitHub/py4e-int/pretext/main.ptx to HTML in /var/folders/l5/mfkvr71161v2zb5_3mrrk8c00000gp/T/ptx-d7dleru2

XSL conversion of /Users/barbarer/Documents/GitHub/py4e-int/pretext/main.ptx by /Users/barbarer/.ptx/2.44.0/core/xsl/pretext-html.xsl

messages from the log for XSL processing:

* PTX:INFO:   checking all deprecated elements.

* failed to load "/Users/barbarer/Documents/GitHub/py4e-int/generated-assets/datafile/MOOC-wikip-id.xml": No such file or directory

error: processing with /Users/barbarer/.ptx/2.44.0/core/xsl/pretext-html.xsl has failed

critical: Cannot resolve URI /Users/barbarer/Documents/GitHub/py4e-int/generated-assets/datafile/MOOC-wikip-id.xml

------------------------

critical: Failed to build without errors.  Exiting...

----------------------------------------------------

While running pretext, the following errors occurred:

(see log messages above or in the 'logs' folder for more information)

error: processing with /Users/barbarer/.ptx/2.44.0/core/xsl/pretext-html.xsl has failed

critical: Cannot resolve URI /Users/barbarer/Documents/GitHub/py4e-int/generated-assets/datafile/MOOC-wikip-id.xml

critical: Failed to build without errors.  Exiting...

----------------------------------------------------

(base) barbarer@Mac pretext % 

[Louis Aloia]

Try running pretext generate -t web to generate the asset and then pretext build web.  You'll also need to wrap the HTML in a CDATA tag to preserve the raw HTML. For example,

<![CDATA[
<html lang="en">
...

</html>
]]>

If you don't want the file contents to appear on the HTML page set the hide attribute to "yes".

  <datafile xml:id="sample-html-id" label="sample-html-id" filename="sample.html" hide="yes">
      <pre source="files/sample.html"></pre>
  </datafile>

[Barbara Ericson]

Thank you!   I was able to get it to work.  I had to edit the HTML file to add the <![CDATA[ at the top and the ]]> at the bottom.