pci 0000:00:01:0: MSI quirk detected; subordinated MSI disabled ...

[Albretch Mueller]

is the error message I get when I try to start a PC based on an MSI
motherboard, with both Debian live an Knoppix (which is a bit more
flexible about start up options)

How can I get passed that error in order to be able to use and
install Debian live or Knoppix?

I have tried a number of options such as acpi=off combined with some
other ones as well as read the MSI-HOWTO.txt

https://elixir.bootlin.com/linux/v2.6.28/source/Documentation/PCI/MSI-HOWTO.txt

on which they talk about "MSI quirks" on chapter 6th, but I can't see
the relationship with booting Debian Live, so I I haven't been able to
fix that problem.

lbrtchx

[David Wright]

You could try disabling what you can in the BIOS to see if you can
boot. Then restore things one by one. That worked for someone—their
problem was caused by USB Legacy Emulation being enabled.

Cheers,
David.

[didier gaumet]

Hello,

I think that this error message is not about MSI-the-manufacturer but
that here MSI stands for Message Signaled Interrupt:
https://www.kernel.org/doc/html/latest/PCI/msi-howto.html

You could then try to boot with the pci=nomsi boot parameter to see if
your PC boots properly. But there is a chance that this message is an
information message about a step of the boot procédure that just has
been succesful and that the problematic step that freezes your computer
is the next one...

[Albretch Mueller]

> You could then try to boot with the pci=nomsi boot parameter ...

if I boot up passing to the kernel the start up option:

knoppix64 debug pci=nomsi noapic

I would get just two lines further bellow:

pci 0000:00:01:0: MSI quirk detected; subordinated MSI disabled
PCI Interrupt Link [LNKA] enabled at IRQ 11
PCI: setting IRQ 11 as level-triggered

and the boot process would stop.

> there is a chance that this message is an information message about a step of the boot procédure that just has been successful and that the problematic step that freezes your computer is the next one...

is there a way for me to know what the next step would be? The kernel
boot up logs should be " (do)ing ....", "... ok"

I know the boot process is different for each motherboard, a bit of
googling gave me:

http://forums.debian.net/viewtopic.php?f=5&t=47164

[ ...] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11
[ ...] ACPI: PCI Interrupt 0000:00:11.1[A] -> Link [LNKA] -> GSI 11
(level, low) -> IRQ 11
[ ...] VIA_IDE 0000:00:11.1: VIA VLink IRQ fixup, from 255 to 11
[ ...] VP_IDE: not 100% native mode: will probe irqs later

so the problem might be related to the video card?

I also found this message at an ubuntu forum:

https://ubuntuforums.org/showthread.php?t=2202279

Re: New PC, cannot install Ubuntu (pci 0000:00:01.0: MSI quirk detected)

Oldfred, I want to thank you for your tireless effort to help me
with resolving this issue. I am very grateful that people like you are
available to the community to support noobs like myself

I am posting this from my new Xubuntu install, so it seems we
finally managed to get things working!!!

Here is a recap from the steps I took that (I think) lead to the resolution:
- (Obviously) go through the posts provided by Oldfred, tried out
several options
- I found some info pointing towards USB legacy support and MSI quirk
- Disabled legacy support in my BIOS
- Got that Checking NVRAM error
- Reset my CMOS (through the jumper setting described in my
motherboard manual)
- Disabled legacy support again, checked some other BIOS settings
to make sure they are what they were before resetting the CMOS
- Set my CD/DVD up as primary boot device (in BIOS)
- Used a (X)Ubuntu installation CD instead of a USB (as legacy
support is now disabled)
- Up till this point I still had that Checking NVRAM error at POST.
- Almost went out to buy a new motherboard, gave it one more try
by unplugging my USB keyboard and mouse, booting with Xubuntu
installation CD (12.04.3 LTS)
- BINGO! Installation starts!!!
- During installation, plug back in my USB keyboard and mouse
- Let it complete
- Restart is needed in order to complete installation
- Had some more issues after that with keyboard not working in GRUB menu
- Somehow magically ended up in my BIOS again (by using the DEL
key during boot)
- Enabled USB legacy support
- Reboot
- Finally able to have Xubuntu boot again, login, install latest
updates, working instllation!

I am hoping that I described all this in enough detail so that it
can help others with similar issues.

In the end it seems that when you get that MSI quirk error,
disabling USB legacy support in your BIOS goes a long way. That
Checking NVRAM error is (or looks to be) because you still have USB
devices plugged in when USB legacy support is disabled.

So resetting the CMOS may not have been necessary after all. I did
not have a PS/2 keyboard/mouse available, but probably disabling
legacy USB support in the BIOS and using PS/2 keyboard+mouse would
have worked directly.
~
but I very much doubt that MSI could be in business requiring so much
work for regular folks out there.

I will keep a log here about how I solved that problem in order to
help people running into the same hurdle

lbrtchx

[didier gaumet]

Le 15/04/2021 à 02:29, Albretch Mueller a écrit :

> if I boot up passing to the kernel the start up option:
>
> knoppix64 debug pci=nomsi noapic
>
> I would get just two lines further bellow:
>
> pci 0000:00:01:0: MSI quirk detected; subordinated MSI disabled
> PCI Interrupt Link [LNKA] enabled at IRQ 11
> PCI: setting IRQ 11 as level-triggered
>
> and the boot process would stop.
>

> is there a way for me to know what the next step would be? The kernel
> boot up logs should be " (do)ing ....", "... ok"

[...]

Hello Albretch,

I do not know if there is a way to deduce what the next step is in the
boot process of a particular hardware. And with systemd, the boot
procedure is parallelized

I am afraid I have no tried and easy solution to propose to you

I do not know what you motherboard is and how old it is.

What I suggest would be to investigate if your BIOS or UEFI has
particular settings for PCI and try to adapt them to the hardware
present and the OS installed/to be installed.

As a further measure, you could try to play with the different kernel
parameters to see if there is an improvement:
https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html

concerned parameters could be:
- *debug* (to increase the level on information you get at boot, but I
doubt it alittle because the Debian kernel is not built with all debug
options in order to be quicker at runtime)
- *irq* (whith the nomsi option, the system would, it seems, be in irq mode)
- *acpi*
- *bios* (to try to forbid the bios to be called by the OS to determine
how to call MSI/IRQ)
- *pci* and *pcie*

As with Knoppix, you can pass a kernel option to the boot in Debian by
hitting "e" key at boot to edit the command line to launch the kernel

Good luck :-)

[Gene Heskett]

Or you could do what I did after fighting with an msi board for several
years a decade back, feed it to the trash can and replace it with an ARK
shoebox w/intel atom D-525-MW motherboad that was just enough power to
get the job done, and Just Worked for a decade. But just in the last 90
days has been replaced with a stack of Dell 7010's with i5 cpu's, off
lease stuff for $150/copy. Much faster stuff, and makes 4 of the 6
running in my farm identical. Simplifies things quite a bit.

Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[Andrew M.A. Cater]

On Thu, Apr 15, 2021 at 06:30:41AM -0400, Gene Heskett wrote:
> On Thursday 15 April 2021 03:50:19 didier gaumet wrote:
>
> > Le 15/04/2021 à 02:29, Albretch Mueller a écrit :
> > > if I boot up passing to the kernel the start up option:
> > >
> > > knoppix64 debug pci=nomsi noapic
> > >
> > > I would get just two lines further bellow:
> > >
> > > pci 0000:00:01:0: MSI quirk detected; subordinated MSI disabled
> > > PCI Interrupt Link [LNKA] enabled at IRQ 11
> > > PCI: setting IRQ 11 as level-triggered
> > >
> > > and the boot process would stop.
> > >
> > > is there a way for me to know what the next step would be? The
> > > kernel boot up logs should be " (do)ing ....", "... ok"
> >

Hello Albretch

Are you still subject to constraints in what you can do with your hardware: I
know a while back in other threads on this mailing list you said that you had
to use virtual machines / live disks.

Is this a desktop / laptop and do you have the ability to change whatever
settings you need?

All the very best, as ever,

Andy C.

[Dan Ritter]

Gene Heskett wrote:
>
> Or you could do what I did after fighting with an msi board for several
> years a decade back, feed it to the trash can and replace it with an ARK

MSI the motherboard manufacturer has nothing to do with the
PCI feature we are discussing here.

https://www.kernel.org/doc/html/latest/PCI/msi-howto.html

-dsr-

[Albretch Mueller]

> What I suggest would be to investigate if your BIOS or UEFI has particular settings for PCI and try
> to adapt them to the hardware present and the OS installed/to be installed.

> As a further measure, you could try to play with the different kernel parameters to see if there is an
> improvement: https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html
> concerned parameters could be:
> - *debug* (to increase the level on information you get at boot, but I doubt it alittle because
> the Debian kernel is not built with all debug options in order to be quicker at runtime)
> - *irq* (whith the nomsi option, the system would, it seems, be in irq mode)
> - *acpi*
> - *bios* (to try to forbid the bios to be called by the OS to determine how to call MSI/IRQ)
> - *pci* and *pcie*

> As with Knoppix, you can pass a kernel option to the boot in Debian by hitting "e" key at boot to edit the command line to launch the kernel

Thank you. I have tried every possible option combination with
knoppix and I have studied reports on similar problems by other
people, but I haven't still been able to make that mobo boot up.

The problem is that the idiots that designed that motherboard make it
hard for you to get to their BIOS (I haven't even figured out that)
and I haven't been able to find their manual/specs for that board
online (most probably for other reasons)

> Or you could do what I did after fighting with an msi board for

> several years a decade back ...

I may have too, but I must work under constraints and on a short
budget and I must be hopeful as well ;-).

> Are you still subject to constraints in what you can do
> with your hardware: I know a while back in other threads
> on this mailing list you said that you had to use virtual
> machines / live disks.

Well, yes. I can only access the internet using live CDs/DVDs
(preferably while using knoppix with the toram option). I have higher
powers actively messing with me. Some people tell me that they "mess
with", monitor everyone that I am just aware and I hate them and their
bs because it messes with my line of work.

I never effing ever connect my work machine to the Internet. I mean,
noticing my doctor telling me whatever with a computer connected to
the Internet makes me anxious and if you think I am "paranoid" all I
have to say is: "thank you". These times being "paranoid" about such
matters is a healthy state of mind.

So, live DVDs with some extra scripts to customize things a bit and I
am fine to go.

I like Debian very much, just using apt in the way that it is
supposed to be used assuming that the Internet is a safe, "trusted"
environment is an odd joke to me. we have had these constant back and
forths in the mailing list. I have even been dreaming about coding a
version of apt based on java (japt?), so that you could go anywhere
with any laptop and get all libraries and etc's, it shouldn't be hard
at all but right now I cannot dedicate time and mind to any other
thing.

>MSI the motherboard manufacturer has nothing to do with the
>PCI feature we are discussing here.

>https://www.kernel.org/doc/html/latest/PCI/msi-howto.html

thank you. it was a confusing coincidence. The thing is that the
motherboard on that box is an:

MSI MS-7641 Ver.3.0 + AMD Athlon II x2 250

the two links were very good but also "too intertaining" I just need
to get the thing to work for me asap.

lbrtchx

[Alexander V. Makartsev]

On 15.04.2021 23:14, Albretch Mueller wrote:

https://www.kernel.org/doc/html/latest/PCI/msi-howto.html

thank you. it was a confusing coincidence. The thing is that the
motherboard on that box is an:

MSI MS-7641 Ver.3.0 + AMD Athlon II x2 250

 the two links were very good but also "too intertaining" I just need
to get the thing to work for me asap.

lbrtchx

Have you tried to update BIOS? [1] Patch notes look promising.
For me it's usually a first step of troubleshooting and resolving possibly hardware related problems.
If first link doesn't lead to support page for your motherboard, you have to find your exact motherboard model using search. [2]
If you flash wrong BIOS image you will brick your motherboard.


[1] https://us.msi.com/Motherboard/support/760GM-P23-FX
[2] https://us.msi.com/search/MS-7641?page=1

-- 
With kindest regards, Alexander.

⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀⠀⠀⠀ 

[didier gaumet]

Le 15/04/2021 à 20:14, Albretch Mueller a écrit :
[...]

> and I haven't been able to find their manual/specs for that board
> online (most probably for other reasons)

[...]

> MSI MS-7641 Ver.3.0 + AMD Athlon II x2 250

[...]

the page of this MB support on the MSI website:
https://www.msi.com/Motherboard/support/760GM-P23-FX#down-bios

the manual:
https://download.msi.com/archive/mnu_exe/M7641v3.0.zip
(hit <DEL> at boot to access the BIOS)

[Albretch Mueller]

Debian doesn't stop to amaze me. I have an "old" 2 GiB RAM
MacAirbook1,1 without its hard drive and without internal CMOS timing
and with only one USB port (I wonder what those folks were thinking
about when they designed those laptops).

A Debian live DVD boots fine from a USB hub, then I go "hwclock --set
..." and do a hot reset (the set time will be kept) and use a USB pen
drive. Knoppix just shows to me a question sign on a light and dark
gray background.

I will get "married" to Debian (why not? some people would marry
their pets ... ;-)) if it includes the following startup options right
of the live DVD:

* toram
* memtest
* testCD

In my opinion none of those functions are hard to include at all.

You can -almost- always go monkey and do that one way or another, but
I would like for Debian to make more official an offline mode for
using apt probably based on java, so that any machine could be used to
download packages to be then installed off-line for those of us who
don't/can't see the Internet as a trusted environment.

I would also love to see networking taken out of the Linux kernel,
but this is an entirely different, hellishly "political" issue.

lbrtchx

[Dan Ritter]

Albretch Mueller wrote:
>
> I would also love to see networking taken out of the Linux kernel,
> but this is an entirely different, hellishly "political" issue.

You can compile your own kernel with no hardware network
drivers.

-dsr-

[Dan Ritter]

Albretch Mueller wrote:


> You can -almost- always go monkey and do that one way or another, but
> I would like for Debian to make more official an offline mode for
> using apt probably based on java, so that any machine could be used to
> download packages to be then installed off-line for those of us who
> don't/can't see the Internet as a trusted environment.

Why? A web browser will work.

http://ftp.debian.org/debian/pool/

Pull the packages you want and save them on your choice of
media, then bring the media over to your off-line box and copy
them over. apt or dpkg can install from the files.

What you lose this way (besides time) is having apt calculate
which supporting packages you need. However,

https://packages.debian.org

will happily tell you all the dependencies of any given package,
and then you can get all the dependencies of each of those, and
so on.

-dsr-

[Darac Marjal]

On 22/04/2021 20:53, Albretch Mueller wrote:
[cut]

> I would also love to see networking taken out of the Linux kernel,
> but this is an entirely different, hellishly "political" issue.

It's not quite the same thing, but you might be able to get what you
want with the Debian HURD port <https://www.debian.org/ports/hurd/>.
HURD is a microkernel whereby the kernel only provides the most basic
services and almost all device drivers are implemented in user space.
>
> lbrtchx
>

[Albretch Mueller]

> What you lose this way (besides time) is having apt calculate
> which supporting packages you need. However,

> https://packages.debian.org

> will happily tell you all the dependencies of any given package,
> and then you can get all the dependencies of each of those, and
> so on.

yes, and "japt" would scrape those pages or even simulate apt to
figure out all dependencies and then download all the packages you
need to install locally, off line.
~

> You can compile your own kernel with no hardware network
> drivers.

Yes, you can! (tm), but imagine, just imagine, as JOhn Lennon sang,
that networking would be taken out of the kernel! At times it amazes
me to discuss with you such issues here. What is the point of using
security based on IP tables when the active code/js bs is used to even
probe the keyboard for the passwords you use in case you use the same
one in your off line and on line machines? They have been using js
injection even as part of general societal AI bots.
~

> HURD is a microkernel whereby the kernel only provides the most basic
> services and almost all device drivers are implemented in user space.

https://www.debian.org/ports/hurd/

https://people.debian.org/~sthibault/failed_packages.txt

https://www.gnu.org/software/hurd/users-guide/using_gnuhurd.html#Networking

Thanks and networking doesn't seem to have been taken out of the
kernel in HURD. My only question would be would HURD make it easier
somehow? It would be a project to definitely look into.

lbrtchx

[Dan Ritter]

Albretch Mueller wrote:
> > What you lose this way (besides time) is having apt calculate
> > which supporting packages you need. However,
>
> > https://packages.debian.org
>
> > will happily tell you all the dependencies of any given package,
> > and then you can get all the dependencies of each of those, and
> > so on.
>
> yes, and "japt" would scrape those pages or even simulate apt to
> figure out all dependencies and then download all the packages you
> need to install locally, off line.

It's a niche need. People who do this at scale keep their own
copies of the entire Debian repository (I do, at work.) People
who do this as one-offs tend not to share your specific needs.

> > You can compile your own kernel with no hardware network
> > drivers.
>
> Yes, you can! (tm), but imagine, just imagine, as JOhn Lennon sang,
> that networking would be taken out of the kernel!

... because for almost everyone, it's better in the kernel.

> At times it amazes
> me to discuss with you such issues here. What is the point of using
> security based on IP tables when the active code/js bs is used to even
> probe the keyboard for the passwords you use in case you use the same
> one in your off line and on line machines? They have been using js
> injection even as part of general societal AI bots.

This paragraph made sense up until the word active, and then it
went right off the rails.

You seem to be conflating three different things.

iptables is a firewall mechanism that allows you to reject
unwanted traffic based on properties of IP packets.

If you're running someone else's malicious code, you have lost
your security game already. Don't do that.

"AI" either doesn't exist or has been around for decades,
depending on how you want to define it. Your "they" is
undefined. JavaScript is just a language; there's nothing
special about it that can't be done in Haskell, Forth or
Fortran. Web browsers tend to implement it, but you can also
turn it off altogether -- e.g. I can think of three different
mechanisms that are practical:

Use Firefox with the Disable JavaScript extension.

Use Firefox with ublock Origin and turn off JavaScript
execution.

Use a non-JavaScript equipped browser.

-dsr-

[Albretch Mueller]

>> Yes, you can! (tm), but imagine, just imagine, as JOhn Lennon sang,
>> that networking would be taken out of the kernel!

>... because for almost everyone, it's better in the kernel.

Here we clearly could agree to disagree on the "for almost everyone"
part, the specifics of the adjetive "better" is what I would like to
know more about. In the same way that Linux defines kernel and user
space, it could also define "inet" space with a specific user for whom
networking is accessible a la SELinux ...

>> At times it amazes
>> me to discuss with you such issues here. What is the point of using
>> security based on IP tables when the active code/js bs is used to even
>> probe the keyboard for the passwords you use in case you use the same
>> one in your off line and on line machines? They have been using js
>> injection even as part of general societal AI bots.
>
>This paragraph made sense up until the word active, and then it
>went right off the rails.
>
>You seem to be conflating three different things.
>
>iptables is a firewall mechanism that allows you to reject
>unwanted traffic based on properties of IP packets.

Well, yes, but IP tables are the easy, unimportant part when it comes
to hacking by government sponsored perpetrators, including your
Internet provider. These days there is no relevant difference
whatsoever between the NSAs of the world and IT companies.

>"AI" either doesn't exist or has been around for decades,
>depending on how you want to define it.

Well, we agree on that to me AI is some bsing pretentious nonsense,
but the NSAs of the world have real-time, click by click access
patterns of every individual of entire societies, to which they apply
Bayesian networks to correlate it intra and intertextually (across
individuals) ... they notice "you are saying something that you
shouldn't be saying" (tm) and your off line and on line life will get
very "interesting" (to put it somehow)

I know it would sound to anyone with some senses spared like stupid
random bs, but they ran me out of the U.S. because they are using
regular people for social control experiments in an MK-Ultra kind of
way, making them believe that they were "reading their mind" (based on
all the data they gather about every single one of us). They even made
that Phillipino president that "God had talked to him". People,
actually wholeheartedly believed such bs. We are talking here about
University professors, ex CIA agents and military personnel, retirees,
with no previous mental illness of any kind not even in their
families, who all of a sudden became schizophrenics in their early
60's (something never heard of and without an explanation
scientifically, neurobiologically) ... people would hear non-stop a
fully, naturally sounding voice (or various voices, they did that bs
to me a few times, but I could tell it was delivered and they stopped)
telling them for example that they had forgotten to take their car
keys once again and teasing them as they looked for them, or that
their spouse was having sex right now with someone else and giving
them their telephone # to call ... most people have committed suicide
and/or gone into their homicidal rampages so common in the U.S. Non of
those "targeted individuals" could make sense of why that was
"happening" to them. You hear the voices tormenting you non-stop and
your spouse sleeping right next to you doesn’t.

I just tried to convince those people that the government could not
read their mind, that they were just using run of the mill techno sh!t
(sensor, their cell phones, all information they consciously or
functionally generate to basically create a doppelgaenger of
themselves ...) to make them believe such nonsense. The FBI/NYPD was
telling me "we are going to have to kill you if you don't stop". The
only way I could make sense of it, is that they needed for people to
believe such bs and talk back to such "voices" as part of their social
control research. The least surface we expose to them the better.

I know most technical folks tend to be kind of cynical, but my point
is that there is more than IPtables we should be worrying about and we
Mathematicians, scientists and tech monkeys of the world are complicit
in creating the state of affairs I am describing. I think we could do
much better than worrying about ip addresses and such things.

I talk about such matters because I think it is better when someone
just twits those issue into general consciousness.

> JavaScript is just a language; there's nothing
>special about it that can't be done in Haskell, Forth or
>Fortran. Web browsers tend to implement it, but you can also
>turn it off altogether

Well many (most? all?) websites including google would not work with
js disabled, so so einfach it is nicht.

lbrtchx

[Celejar]

On Fri, 23 Apr 2021 15:43:12 -0400
Dan Ritter <d...@randomstring.org> wrote:

...

> undefined. JavaScript is just a language; there's nothing
> special about it that can't be done in Haskell, Forth or
> Fortran. Web browsers tend to implement it, but you can also
> turn it off altogether -- e.g. I can think of three different
> mechanisms that are practical:
>
> Use Firefox with the Disable JavaScript extension.
>
> Use Firefox with ublock Origin and turn off JavaScript
> execution.
>
> Use a non-JavaScript equipped browser.

Or toggle "javascript.enabled" to false in "about:config" - no
extensions required.

Celejar

[Albretch Mueller]

> Or toggle "javascript.enabled" to false in "about:config" - no
> extensions required.

the assumption being that you would use, trust firefox. A
well-documented, XML-based open source kind of proxy/gateway through
which all requests are sent and received would work with any browser.
~
I think, given the options, the suggested idea not to install
networking at all (one of the install options with Linux) wins the
prize; then:

b) an exposed "inet" user for the session should be defined with a
RAM drive as its home directory;
c) for whom -exclusively- the network card's device driver should be
installed (is here a hack to install a device driver entirely from
user space? probably, a Debian HURD kind of thing on which the basic
hook for the networking service is exposed and a user space program
would take it from there?);
d) then all access to the Internet go through a client proxy run by "inet";
e) acting as a squid-like application proxy;
e.1) parsing all text/html content coming in, first cleansing it into
well-fomed xhtml with some HTML parser, then using XPath, java
Nasshorn, the principles of the Burp Suite + wireshark + page Reader +
...;
e.1.1) sanitzing the page into alpha-offset format;
e.1.2) not the received, but a sanitzed, topical, content centric
page with all the distracting "attentional" nonsense removed will be
displayed to the user (page reader on steroids);
e.1.3) it maintain a local cache of the accessed data (videos, pdf
files, ...) and adjusts all links on the flight to the local paths
(storage space is very cheap, anyway);
e.1.4) using XPaths of the sanitized page, you would be able to
exactly authorize the java script that you want to let through and
blanket "yeah, sure!" the rest of it, including those bsing "we care
about your privacy" (isn't that the very definition of not having
privacy that other people "care" about it?), "user agreements"
authorization of cookies, right on the proxy ebfore it reaches your
field of view/consciousness with the option of sending bogus
information actively, passively or by request, per site, section of
the page ...;
e.1.5) cookies, user agent, ... will be managed by the proxy, so
little as possible will be exposed;
f) based on the request headers the proxy would create a new
folder/jail each time a new domain is accessed (probably even the
option to run macchanger?);
g) based on the critical XPaths a "processing signature" will be kept
for pages, domains, URL paths frequented;
h) inet had no access to the PATH or any other local variables or
files, it would only know of its own jail;
i) many/most/all? applications assume/demand Internet access. All
such access attempts should be logged with traceable information so
that you can inspect what it is "that application was attempting to do
which it shouldn't be doing" (tm);
j) while shutting down "inet" will permanently save the cookies and
such which matter, the rest will be physically "hasta la vista,
babied" since it was kept in RAM anyway;
k) that proxy could be used as password jar and could as well work as
agent regularly checking if you've got new or certain kinds of email
...

These is nothing really "new" in that spec. All the pieces either
exist or their functional requirements can be achieved with
configuration changes. It is assembling of all pieces which matters
and may be problematic.

Do you know of such previous "paranoid" art?

Any Linux/debian friendly network firmware you would recommend to
make that project easier?

Any ideas come to mind? Any hints or death threats you would share?

lbrtchx

[davidson]

On Fri, 23 Apr 2021 Albretch Mueller wrote:

On Thu 22 Apr 2021 Dan Ritter wrote:
>> What you lose this way (besides time) is having apt calculate
>> which supporting packages you need. However,
>
>> https://packages.debian.org
>
>> will happily tell you all the dependencies of any given package,
>> and then you can get all the dependencies of each of those, and
>> so on.
>
> yes, and "japt" would scrape those pages or even simulate apt to
> figure out all dependencies and then download all the packages you
> need to install locally, off line.
> ~

How similar, to the offline machine that will install the packages, is
the machine that connects to the internet to download them ?

If sufficiently similar, there is the '-d' option (--download-only)
for apt-get.

--
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

[davidson]

What is "alpha-offset format"?

[davidson]

On Thu, 22 Apr 2021 Albretch Mueller wrote:
[snip]

>
> I will get "married" to Debian (why not? some people would marry
> their pets ... ;-)) if it includes the following startup options right
> of the live DVD:
>
> * toram
> * memtest
> * testCD
>
> In my opinion none of those functions are hard to include at all.
>
> You can -almost- always go monkey and do that one way or another, but
> I would like for Debian to make more official an offline mode for
> using apt probably based on java, so that any machine could be used to
> download packages to be then installed off-line for those of us who
> don't/can't see the Internet as a trusted environment.

I haven't seen anyone mention apt-offline in this thread, so I will
mention it now:

apt-offline is an Offline APT Package Manager.
.
apt-offline can fully update and upgrade an APT based distribution
without connecting to the network, all of it transparent to APT.
.
apt-offline can be used to generate a signature on a machine (with
no network). This signature contains all download information
required for the APT database system. This signature file can be
used on another machine connected to the internet (which need not be
a Debian box and can even be running windows) to download the
updates. The downloaded data will contain all updates in a format
understood by APT and this data can be used by apt-offline to update
the non-networked machine.
.
apt-offline can also fetch bug reports and make them available
offline.
Homepage: https://github.com/rickysarraf/apt-offline

I have no experience using it. Looks to me like for buster you need
to get it from backports.

[Dan Ritter]

davidson wrote:
> > I would like for Debian to make more official an offline mode for
> > using apt probably based on java, so that any machine could be used to
> > download packages to be then installed off-line for those of us who
> > don't/can't see the Internet as a trusted environment.
>
> I haven't seen anyone mention apt-offline in this thread, so I will
> mention it now:

Thanks! I didn't know about it.

-dsr-

[Albretch Mueller]

> What is "alpha-offset format"?

we, corpora research kinds of folks, need to process thousand of
files as other people process bytes. UTF8 was basically an
Americanizierung of alle alphabets. UTF is great to describe an
alphabet but not for text files.

UTF8 turned all files into streams not good for questions such as
what is the charatcer/string sequence starting on the nth addressable
unit of a file ...

Doing that with utF8 is from way too complicated to impossible. Also
alpha offset nicely splits the files segments into its different
parts: ALPHABETICAL text, js, css, ...

lbrtchx

[Darac Marjal]

On 29/04/2021 14:03, Albretch Mueller wrote:
>> What is "alpha-offset format"?
> we, corpora research kinds of folks, need to process thousand of
> files as other people process bytes. UTF8 was basically an
> Americanizierung of alle alphabets. UTF is great to describe an
> alphabet but not for text files.
>
> UTF8 turned all files into streams not good for questions such as
> what is the charatcer/string sequence starting on the nth addressable
> unit of a file ...

Depends on what you mean by "addressable unit", surely? UTF8 is a
variable-length record format, but it's still addressable. Basically,
it's like taking a CSV file and saying "what's the contents of the cell
starting at byte 123"? CSV cells are variable length. Perhaps there
isn't such a cell. If you want to know the contents of the cell which
includes byte 123, then you need some context, don't you?

>
> Doing that with utF8 is from way too complicated to impossible. Also
> alpha offset nicely splits the files segments into its different
> parts: ALPHABETICAL text, js, css, ...

So, do you use something more like UTF-32?
>
> lbrtchx
>

[rhkr...@gmail.com]

Ok, but what does it look like? (What is the format?)

A google search shows only links to this thread and some page about its
relevance to aiming a telescope -- I strongly suspect that is not relevant to
your use case.

[David Wright]

On Thu 29 Apr 2021 at 11:33:29 (-0400), rhkr...@gmail.com wrote:
> On Thursday, April 29, 2021 09:03:59 AM Albretch Mueller wrote:
> > > What is "alpha-offset format"?
> >
> > we, corpora research kinds of folks, need to process thousand of
> > files as other people process bytes. UTF8 was basically an
> > Americanizierung of alle alphabets.

"Americanizierung" merely seems an odd way of saying that UTF8 favours
the Roman alphabet in terms of the length of encoded strings.

> > UTF is great to describe an
> > alphabet but not for text files.

Are we now discussing non-alphabetic scripts? Beats me.

> > UTF8 turned all files into streams not good for questions such as
> > what is the charatcer/string sequence starting on the nth addressable
> > unit of a file ...

It's very good for such questions, because it is unambiguous whether
the nth addressable unit (let's say byte) is the start of a character
or not. When it /is/ the start of a character, the number of bytes in
the character is given by at most 1–5 bits of that first byte, so you
don't have to hunt for some kind of terminating byte.

> > Doing that with utF8 is from way too complicated to impossible.

OTOH it's not straightforward to determine at which byte does the
nth character start. Perhaps that is what you meant to say.

> > Also
> > alpha offset nicely splits the files segments into its different
> > parts: ALPHABETICAL text, js, css, ...

I'd be interested to see examples of JavaScript and Cascading Style
Sheets written in non-ALPHABETICAL text.

> Ok, but what does it look like? (What is the format?)
>
> A google search shows only links to this thread and some page about its
> relevance to aiming a telescope -- I strongly suspect that is not relevant to
> your use case.

Whatever it is is not well described by that post, but perhaps
better by (e.1.2) immediately following it.

>From the context, alpha-offset format sounds like jargon for what
html2text does with a web page, which is to separate the text you
want to read from all the interspersed code that controls how it
would be displayed by a browser.

That text¹, in whatever encoding, and now called "corpora", could be
analysed as part of someone's corpus linguistics research.

The major problem that seems to be exercising this thread at this
point is that the OP needs to swallow scads of documents, some, many,
perhaps most of which are on the web, but process them on a computer
that has never been connected to the internet.

Some of us, decades ago, were coping with this problem, as it applies
to the software, in the days before Debian's tools had evolved to the
functionality they now have. These tools seem to have been overlooked.

And so we have discussions of removing networking from the kernel,
redesigning apt using java, and de-americanising unicode, all
closely monitored, keystroke by keystroke, by NSAs around the world.
Typical debian-user, eh? Did we ever decide which meaning of MSI
applied for explaining the quirk in the subject line? Perhaps
Randolph Quirk would coincidentally have held an opinion.

¹ I assume a similar treatment is meted out to PDFs, and the cache
of videos is fed into a speech analyser, all to add to the mountain
of text.

Cheers,
David.

[Albretch Mueller]

> I haven't seen anyone mention apt-offline in this thread, so I will
> mention it now:
>
> apt-offline is an Offline APT Package Manager.
> .
> apt-offline can fully update and upgrade an APT based distribution
> without connecting to the network, all of it transparent to APT.
> .
> apt-offline can be used to generate a signature on a machine (with
> no network). This signature contains all download information
> required for the APT database system. This signature file can be
> used on another machine connected to the internet (which need not be
> a Debian box and can even be running windows) to download the
> updates. The downloaded data will contain all updates in a format
> understood by APT and this data can be used by apt-offline to update
> the non-networked machine.
> .
> apt-offline can also fetch bug reports and make them available
> offline.
> Homepage: https://github.com/rickysarraf/apt-offline
>
> I have no experience using it. Looks to me like for buster you need
> to get it from backports.

thank you very much. I see myself using it intensively to deal with
my kinds of problems.
the best guideline I found at:

https://sleeplessbeastie.eu/2014/01/30/how-to-manage-packages-on-an-off-line-debian-system/

about the alpha-offset encoding, please, givie me some time to answer
your questions, write up the idea more fully, clearly and I think we
should probably start a new thread to discuss it

lbrtchx

[davidson]

On Thu, 29 Apr 2021 Albretch Mueller wrote:
>> What is "alpha-offset format"?
>
> we, corpora research kinds of folks, need to process thousand of
> files as other people process bytes.

That was a helpful clue, that it could be a term of art in corpus
linguistics.

After some searches in that direction, I found a couple of papers
using the term "character offset format", which sounds a little more
familiar.

> UTF8 was basically an Americanizierung of alle alphabets.

All roads lead to [DEL: Rome :DEL] American Standard Code for
Information Interchange.

> UTF is great to describe an alphabet but not for text files.
>
> UTF8 turned all files into streams not good for questions such as
> what is the charatcer/string sequence starting on the nth
> addressable unit of a file ...

Variable-width encoding is a complication if you need random access to
the nth character. I see.

> Doing that with utF8 is from way too complicated to impossible.

The solution, clearly, is for everyone to use UTF-32 instead.

> Also alpha offset nicely splits the files segments into its
> different parts: ALPHABETICAL text, js, css, ...

Ah. It sets *character data* apart from markup, annotations, etc.

Definition: All text that is not markup constitutes the "character
data" of the document. ( https://www.w3.org/TR/REC-xml/#dt-chardata )

An alpha/character offset format separates the data from things said
*about* the data.

If I understand you correctly, that is.

Now I wonder how this might enable random access to the nth
character. I will keep looking around.

Anyways, thank you for taking the time to elaborate.

And good luck with your projects. Many of the points in your outline
here

https://lists.debian.org/msgid-search/CAFakBwg8yrCe1MFvK91_Ovu6HOO24cz=rm101mbcDBLZ0=TU...@mail.gmail.com

particularly the ones under section

> e) acting as a squid-like application proxy

are on my wishlist, because I think a web browser should resemble more
a pair of binoculars than the eyelid retainers in Stanley Kubrick's A
Clockwork Orange.

[Stefan Monnier]

> Now I wonder how this might enable random access to the nth
> character. I will keep looking around.

Another part of the question is: why would someone give you the position
information in terms of characters rather than in terms of (say) bytes,
or words, or ...


Stefan

[davidson]

On Thu, 29 Apr 2021 Albretch Mueller wrote:
[dd]

>
> about the alpha-offset encoding, please, givie me some time to answer
> your questions,

Of course. Whenever. No hurry.

> write up the idea more fully, clearly

I won't discourage anyone determined to explain the tools of their
specialty to the layman. That would be very cool of you.

But a pointer to some description/specification of such a format would
work too, if that turns out to be convenient.

> and I think we should probably start a new thread to discuss it

Heh. Probably true.

[David Wright]

I thought we'd scotched bytes, because the number of bytes in a
character depends on the encoding: the simple letter A occupies
1 byte in ASCII and UTF-8, 2 bytes in UTF-16, and 4 in UTF-32.

> or words,

Counting words raises the problem of compound words, because people
spell them differently: as separate words with spaces, or hyphenated,
or as a single word. In English, compound words tend to evolve in
that order, as we grow comfortable with seeing the words joined
together. So the same sentence might have fewer words in it just
because it was written in more modern times.

> or ...

It's fairly usual to count characters, ignoring whitespace
and punctuation. There are still complications for languages
like Welsh, where there are single letters like ch and ll.
I remember, as a small child on holiday, learning to spell,
pronounce and translate the 58-letter place name
Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch
and not realising for years that it was only 51 letters long,
and didn't really contain a 4-letter repeat.

BTW I, too, naturally googled A-O F, and found the same telescopic
reference as rhkramer, though looking once again, I see this thread is
starting to add far more hits for the term (which I won't repeat here).

Cheers,
David.