Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

The Register reprises arguments about use of AI in legal work

62 views
Skip to first unread message

Nick Odell

unread,
Jan 10, 2024, 11:20:11 AM1/10/24
to
https://www.theregister.com/2024/01/10/top_large_language_models_struggle/

Very little here that wasn't discussed in ulm passim but may be of
interest. Some of the BTL comments are worth a read (but some of them
are not!)

Nick

Pamela

unread,
Jan 10, 2024, 1:23:30 PM1/10/24
to
On 16:20 10 Jan 2024, Nick Odell said:
>
> https://www.theregister.com/2024/01/10/top_large_language_
> models_struggle/
>
> Very little here that wasn't discussed in ulm passim but may be of
> interest. Some of the BTL comments are worth a read (but some of them
> are not!)
>
> Nick
>

"Last year, when OpenAI showed GPT-4 was capable of passing the Bar
Exam, it was heralded as a breakthrough in AI and led some people to
question whether the technology could soon replace lawyers."

What score is required to pass the Bar Exam?

In some professions, such as accountancy, the pass mark is 50% to 55%
depending on which paper you are. That leaves a lot of room for
ignorance.

Simon Parker

unread,
Jan 15, 2024, 7:34:29 PM1/15/24
to
On 10/01/2024 16:32, Pamela wrote:
> On 16:20 10 Jan 2024, Nick Odell said:
>>
>> https://www.theregister.com/2024/01/10/top_large_language_
>> models_struggle/
>>
>> Very little here that wasn't discussed in ulm passim but may be of
>> interest. Some of the BTL comments are worth a read (but some of them
>> are not!)
>
> "Last year, when OpenAI showed GPT-4 was capable of passing the Bar
> Exam, it was heralded as a breakthrough in AI and led some people to
> question whether the technology could soon replace lawyers."
>
> What score is required to pass the Bar Exam?

It varies. The Uniform Bar Exam (UBE) has a maximum score of 400. In
some states a UBE score of 266 would be considered a pass whilst in
other states it needs to be 280 or above.


> In some professions, such as accountancy, the pass mark is 50% to 55%
> depending on which paper you are. That leaves a lot of room for
> ignorance.

GPT-4 didn't just "pass" the UBE, it was in the 90th percentile.

However, it must be borne in mind that ChatGPT uses the GPT-3.5 series
model which failed the UBE.

Regards

S.P.

Jon Ribbens

unread,
Jan 15, 2024, 7:52:34 PM1/15/24
to
Of course, answering a series of questions which are highly likely to
have been asked and answered many times in the past, i.e. an exam, is
exactly the sort of thing that can be answered by anyone or anything
with excellent recall and little to no intelligence.

By chance I happen to have earlier today asked Google's and Microsoft's
AI implementations some questions regarding a trivia question which
I suspect has been seldom asked before: "what English football team has
the name with the largest number of words?".

They both said that the answer was "AFC Bournemouth" - Google because
that's 4 words, and Microsoft because that's 3 words. Other teams such
as "Arsenal Football Club" and "Blackburn Rovers Football Club" are out
because clearly they have only 2 words each in their respective names.

After I pointed out to Microsoft's AI that its answer was nonsense,
it apologised and said the correct answer was "AFC Wimbledon", because
that is obviously longer than "AFC Bournemouth".

Simon Parker

unread,
Jan 17, 2024, 4:24:24 AM1/17/24
to
On 16/01/2024 00:52, Jon Ribbens wrote:
> On 2024-01-16, Simon Parker <simonpa...@gmail.com> wrote:
>> On 10/01/2024 16:32, Pamela wrote:

>>> What score is required to pass the Bar Exam?
>>
>> It varies. The Uniform Bar Exam (UBE) has a maximum score of 400. In
>> some states a UBE score of 266 would be considered a pass whilst in
>> other states it needs to be 280 or above.
>>
>>> In some professions, such as accountancy, the pass mark is 50% to 55%
>>> depending on which paper you are. That leaves a lot of room for
>>> ignorance.
>>
>> GPT-4 didn't just "pass" the UBE, it was in the 90th percentile.
>>
>> However, it must be borne in mind that ChatGPT uses the GPT-3.5 series
>> model which failed the UBE.
>
> Of course, answering a series of questions which are highly likely to
> have been asked and answered many times in the past, i.e. an exam, is
> exactly the sort of thing that can be answered by anyone or anything
> with excellent recall and little to no intelligence.

Well, GPT3.5 (currently in use in ChatGPT) didn't just fail the UBE but
was in the bottom 10th percentile.

I can't help but wonder if specific very targetted work was done between
the series 3.5 and 4 models to assist GPT-4 in passing the UBE thus
enabling this to be announced to much fanfare.


> By chance I happen to have earlier today asked Google's and Microsoft's
> AI implementations some questions regarding a trivia question which
> I suspect has been seldom asked before: "what English football team has
> the name with the largest number of words?".
>
> They both said that the answer was "AFC Bournemouth" - Google because
> that's 4 words, and Microsoft because that's 3 words. Other teams such
> as "Arsenal Football Club" and "Blackburn Rovers Football Club" are out
> because clearly they have only 2 words each in their respective names.

Pedantically, both AIs are correct but they don't know why and so cannot
provide context or explain their answer which leaves you none the wiser.

In 1923 Boscombe Football Club was renamed to "Bournemouth & Boscombe
Athletic Football Club" which is 7 words and therefore the English
Football team with the largest number of words in its name.

However, in 1971 it adopted the name "AFC Bournemouth" so the answer was
correct only between 1923 and 1971, but as you didn't ask "which
*current* English football team..." I will propose that as a good
candidate for the correct answer.

If considering only current club names, thereby excluding historic ones,
I would suggest that the answer is "Brighton & Hove Albion Football
Club" with six words.

Tangentially, Arsenal is an interesting club name as it is only one of
two professional clubs in England not named after a specific area. A
bonus point if you know the other. :-)


> After I pointed out to Microsoft's AI that its answer was nonsense,
> it apologised and said the correct answer was "AFC Wimbledon", because
> that is obviously longer than "AFC Bournemouth".

Oh dear!

Regards

S.P.

billy bookcase

unread,
Jan 17, 2024, 1:28:41 PM1/17/24
to

"Simon Parker" <simonpa...@gmail.com> wrote in message
news:l0pkdp...@mid.individual.net...
Er

Brighton & Hove Albion Football Club ?

https://en.wikipedia.org/wiki/Brighton_%26_Hove_Albion_F.C.



>
> If considering only current club names, thereby excluding historic ones, I would
> suggest that the answer is "Brighton & Hove Albion Football Club" with six words.
>
> Tangentially, Arsenal is an interesting club name as it is only one of two professional
> clubs in England not named after a specific area. A bonus point if you know the other.
> :-)


Port Vale is usually given as the answer although according to Wiki they're
named after the valley of ports on the Trent and Mersey Canal.

bb

GB

unread,
Jan 17, 2024, 2:10:21 PM1/17/24
to
On 17/01/2024 09:19, Simon Parker wrote:

> Tangentially, Arsenal is an interesting club name as it is only one of
> two professional clubs in England not named after a specific area.  A
> bonus point if you know the other. :-)

I always assumed Arsenal was named after Woolwich Arsenal? Have I been
wrong all these years?


If the question asked was really "what English football team has
the name with the largest number of words?", I don't understand why some
local team wasn't mentioned, for example:
Wingate & Finchley (Youth) Under 16 Eastern Junior Alliance Team








Roland Perry

unread,
Jan 17, 2024, 2:26:26 PM1/17/24
to
In message <uo98ml$25n4b$1...@dont-email.me>, at 19:10:14 on Wed, 17 Jan
2024, GB <NOTso...@microsoft.invalid> remarked:
>On 17/01/2024 09:19, Simon Parker wrote:
>
>> Tangentially, Arsenal is an interesting club name as it is only one
>>of two professional clubs in England not named after a specific
>>area.  A bonus point if you know the other. :-)
>
>I always assumed Arsenal was named after Woolwich Arsenal? Have I been
>wrong all these years?

According to Wonkypedia: "In October 1886, Scotsman David Danskin and
fifteen fellow munitions workers in Woolwich formed Dial Square Football
Club, named after a workshop at the heart of the Royal Arsenal complex.
Each member contributed sixpence and Danskin also added three shillings
to help form the club. The club had renamed to Royal Arsenal by January
1887"

>If the question asked was really "what English football team has
>the name with the largest number of words?", I don't understand why
>some local team wasn't mentioned, for example:
>Wingate & Finchley (Youth) Under 16 Eastern Junior Alliance Team

The question-answering gadget is probably adding some invisible words
like "professional" or "member of the league".
--
Roland Perry

billy bookcase

unread,
Jan 17, 2024, 2:34:15 PM1/17/24
to

"GB" <NOTso...@microsoft.invalid> wrote in message news:uo98ml$25n4b$1...@dont-email.me...
> On 17/01/2024 09:19, Simon Parker wrote:
>
>> Tangentially, Arsenal is an interesting club name as it is only one of two
>> professional clubs in England not named after a specific area. A bonus point if you
>> know the other. :-)
>
> I always assumed Arsenal was named after Woolwich Arsenal? Have I been wrong all these
> years?

No you are correct.

The Woolwich Arsenal was indeed a place, indeed a place of work
from which the team originated. However for their first matches they
called themselves "Dial Square" after the sundial at the top of the
factory gates.

While the rest, as they say, is history..


> If the question asked was really "what English football team has
> the name with the largest number of words?", I don't understand why some local team
> wasn't mentioned, for example:
> Wingate & Finchley (Youth) Under 16 Eastern Junior Alliance Team

er

Wingate & Finchley (Youth) Under 16 Eastern Junior Alliance Team Reserves ?


bb





Jon Ribbens

unread,
Jan 17, 2024, 4:49:53 PM1/17/24
to
On 2024-01-17, GB <NOTso...@microsoft.invalid> wrote:
> On 17/01/2024 09:19, Simon Parker wrote:
>> Tangentially, Arsenal is an interesting club name as it is only one of
>> two professional clubs in England not named after a specific area.  A
>> bonus point if you know the other. :-)
>
> I always assumed Arsenal was named after Woolwich Arsenal? Have I been
> wrong all these years?

It is, but I think the idea is that after you remove Woolwich from the
team name, and remove the team from Woolwich, the team is no longer named
after a place.

> If the question asked was really "what English football team has
> the name with the largest number of words?", I don't understand why some
> local team wasn't mentioned, for example:
> Wingate & Finchley (Youth) Under 16 Eastern Junior Alliance Team

Most people asking trivia questions about football teams will be talking
about some specific subset of all conceivable teams that I don't know
enough about football to describe, e.g. Premier League plus Football
League, or something. So "AI", which is merely mixing and regurgitating
things that people have already said, will be working on the same basis.

Mark Goodge

unread,
Jan 17, 2024, 5:06:52 PM1/17/24
to
On Wed, 17 Jan 2024 19:10:14 +0000, GB <NOTso...@microsoft.invalid> wrote:

>On 17/01/2024 09:19, Simon Parker wrote:
>
>> Tangentially, Arsenal is an interesting club name as it is only one of
>> two professional clubs in England not named after a specific area.  A
>> bonus point if you know the other. :-)
>
>I always assumed Arsenal was named after Woolwich Arsenal? Have I been
>wrong all these years?

Historically, yes. But the question relates to current usage. Arsenal have
gone through several names, including Dial Square, Royal Arsenal and
Woolwich Arsenal before becoming just plain Arsenal.

A lot of football clubs have changed their names over the years, and many
that are now named after their location were originally not. Southampton FC
started life as St Mary's YMA FC, after the church where it started (a name
now reflected in the name of their stadium), and Manchester City likewise
started out as a church team, in their case St Mark's. Everton were
originally St Domingo FC, for similar reasons.

Since nobody else has, I'll claim the bonus point by saying that the other
English professional club not named after their location is Port Vale.
Which, incidentally, is also the subject of a totally different triva
factlet: Port Vale hold the record for the most seasons in the Football
league without ever reaching the top tier.

Port Vale is also a reason why the originally stated triva challenge must
relate to current usage, because Port Vale, like Arsenal but unlike Everton,
have gone the other way, being originally known as Burslem Port Vale before
dropping the town name in 1907.

Mark

GB

unread,
Jan 18, 2024, 5:07:57 AM1/18/24
to
On 17/01/2024 21:49, Jon Ribbens wrote:

>> If the question asked was really "what English football team has
>> the name with the largest number of words?", I don't understand why some
>> local team wasn't mentioned, for example:
>> Wingate & Finchley (Youth) Under 16 Eastern Junior Alliance Team
>
> Most people asking trivia questions about football teams will be talking
> about some specific subset of all conceivable teams that I don't know
> enough about football to describe, e.g. Premier League plus Football
> League, or something. So "AI", which is merely mixing and regurgitating
> things that people have already said, will be working on the same basis.
>

I accept that Chat GPT is a giant cut and paste plagiarism machine.
Nevertheless, I was surprised that it didn't:
a) Parse the question to work out that what was needed was to go through
a list of all football clubs stored in memory
b) count the number of words in each name.

Of course, thinking about it, it can't possibly do such a sequential
search through its vast database in any reasonable time and at a
reasonable cost, so it must be doing an indexed search of some sort, and
that seems to be restricted to other people's answers to the question.

Simon Parker

unread,
Jan 18, 2024, 7:04:52 AM1/18/24
to
On 17/01/2024 10:07, billy bookcase wrote:
> "Simon Parker" <simonpa...@gmail.com> wrote in message
> news:l0pkdp...@mid.individual.net...
>> On 16/01/2024 00:52, Jon Ribbens wrote:

[...]

>>> By chance I happen to have earlier today asked Google's and Microsoft's
>>> AI implementations some questions regarding a trivia question which
>>> I suspect has been seldom asked before: "what English football team has
>>> the name with the largest number of words?".
>>>
>>> They both said that the answer was "AFC Bournemouth" - Google because
>>> that's 4 words, and Microsoft because that's 3 words. Other teams such
>>> as "Arsenal Football Club" and "Blackburn Rovers Football Club" are out
>>> because clearly they have only 2 words each in their respective names.
>>
>> Pedantically, both AIs are correct but they don't know why and so cannot provide
>> context or explain their answer which leaves you none the wiser.
>>
>> In 1923 Boscombe Football Club was renamed to "Bournemouth & Boscombe Athletic Football
>> Club" which is 7 words and therefore the English Football team with the largest number
>> of words in its name.
>>
>> However, in 1971 it adopted the name "AFC Bournemouth" so the answer was correct only
>> between 1923 and 1971, but as you didn't ask "which *current* English football team..."
>> I will propose that as a good candidate for the correct answer.
>
> Er
>
> Brighton & Hove Albion Football Club ?
>
> https://en.wikipedia.org/wiki/Brighton_%26_Hove_Albion_F.C.

If only I hadn't mentioned that in the post, in the very next paragraph
even...


>> If considering only current club names, thereby excluding historic ones, I would
>> suggest that the answer is "Brighton & Hove Albion Football Club" with six words.
>>
>> Tangentially, Arsenal is an interesting club name as it is only one of two professional
>> clubs in England not named after a specific area. A bonus point if you know the other.
>> :-)
>
>
> Port Vale is usually given as the answer although according to Wiki they're
> named after the valley of ports on the Trent and Mersey Canal.

The club's official web-site [1] disagrees and I expect the club to know
the origin of the club better than Wikipedia, therefore I'll stick with
what the club say, (which for anyone interested is): "Port Vale Football
Club was formed in 1876 and took its name from the venue of the
inaugural meeting at 'Port Vale House' situated in a suburb of
Stoke-on-Trent. Upon moving to Burslem in 1884 the Club changed its
name to 'Burslem Port Vale' and after several seasons in the Midland
League became founder members of the Football League Division Two in 1892."

Regards

S.P.

[1] https://www.port-vale.co.uk/history/

Simon Parker

unread,
Jan 18, 2024, 7:05:36 AM1/18/24
to
Wingate & Finchley *Women's* (Youth) Under 16 Eastern Junior Alliance
Team Reserves. :-)

Regards

S.P.

billy bookcase

unread,
Jan 18, 2024, 8:36:09 AM1/18/24
to

"Simon Parker" <simonpa...@gmail.com> wrote in message
news:l0sieo...@mid.individual.net...
> On 17/01/2024 10:07, billy bookcase wrote:
>>
>> Brighton & Hove Albion Football Club ?
>>
>> https://en.wikipedia.org/wiki/Brighton_%26_Hove_Albion_F.C.
>
> If only I hadn't mentioned that in the post, in the very next
> paragraph even...
>
>
>>> If considering only current club names, thereby excluding historic ones,
>>> I would suggest that the answer is "Brighton & Hove Albion Football Club"
>>> with six words.

Ooops!



>>>
>>> Tangentially, Arsenal is an interesting club name as it is only one of two
>>> professional
>>> clubs in England not named after a specific area. A bonus point if you know >>> the
>>> other.
>>> :-)
>>
>>
>> Port Vale is usually given as the answer although according to Wiki they're
>> named after the valley of ports on the Trent and Mersey Canal.
>
> The club's official web-site [1] disagrees and I expect the club to
> know the origin of the club better than Wikipedia, therefore I'll
> stick with what the club say, (which for anyone interested is):

> "Port Vale Football Club was formed in 1876 and took its name
> from the venue of the inaugural meeting at 'Port Vale House' situated
> in a suburb of Stoke-on-Trent. Upon moving to Burslem in 1884 the Club
> changed its name to 'Burslem Port Vale' and after several seasons in
> the Midland League became founder members of the Football League Division
> Two in 1892."
>
> [1] https://www.port-vale.co.uk/history/

Well yes. But then that simply begs the question as to the origin of
the "Port Vale" in "Port Vale House". With both Port Vale House,
and subsequently Port Vale F.C being named after this elusive
Port Vale


It seems there are a number of theories, including solid evidence
in the form of bricks with the lettering "Burslem Port Vale"
and a pre-existing Port Vale Cricket Club; however all of them
still beg the question as to the origin of the term "Port Vale"
itself.

About the only explanation offered for this appears to be as follows

quote:

The accepted theory comes from respected Potteries historian Jeff
Kent whose research found that the club was probably formed in 1879
as an offshoot of Porthill Victoria FC and took its name from the
valley of canal ports where the team played.

:unquote


https://www.onevalefan.co.uk/2020/01/why-called-port-vale/


Which is the basis of the Wikipedia explanation.

And is presumably where Port Vale House got the name as well.



bb






>






Simon Parker

unread,
Jan 18, 2024, 11:34:57 AM1/18/24
to
The bonus point is yours. You can either use it to obtain the warm all
over glow that comes from being right, or save them up throughout 2024
when you may be able to exchange them for anything from a whisky tumbler
to a hi-fi. (Any resemblance to Esso Tiger Tokens is completely
coincidental.)

Regards

S.P.

Mark Goodge

unread,
Jan 18, 2024, 11:49:50 AM1/18/24
to
On Thu, 18 Jan 2024 13:35:56 -0000, "billy bookcase" <bi...@anon.com> wrote:

>quote:
>
>The accepted theory comes from respected Potteries historian Jeff
>Kent whose research found that the club was probably formed in 1879
>as an offshoot of Porthill Victoria FC and took its name from the
>valley of canal ports where the team played.
>
>:unquote

There was a Port Vale Street (which still exists) and a Port Vale Wharf
(which doesn't any more, but is well documented) in the local area. There
was also the Port Vale Cricket Club, which predated the football club. And
"Port Vale" bricks were produced by several local brickworks[1] So even if
there's no evidence of a "Port Vale House" (which is only ever mentioned in
histories of the football club, never anywhere else), the name itself
certainly existed.

Also, the choice of "Port Vale" as a punning alternative to "Porthill" is
fairly obvious. So it's likely that the founders of the football club
deliberately picked a name which both reflected their local area and had a
humorous nod to the club they'd started out at.

[1] https://www.thepotteries.org/brickworks/midland.htm

Mark

0 new messages