How does incoming caller ID work - and more specifically - can caller ID source data be controlled?

[Arlen Holder]

Tough question... likely only for Android freeware experts...

How does incoming caller ID work and more specifically
is there a method you know of to CONTROL its source data?

Essentially, the goal is to shift incoming CallerID off of the
system internal default contacts sqlite database onto a
given privacy-oriented contacts database of your own choosing.
--
Attaining PRIVACY on a mobile device requires knowledge & tools.

[Gary R. Schmidt]

On 01/06/2020 05:39, Arlen Holder wrote:
> Tough question... likely only for Android freeware experts...
>

> How does incoming "Calling Number Identification" work and more specifically

> is there a method you know of to CONTROL its source data?
>
> Essentially, the goal is to shift incoming CallerID off of the
> system internal default contacts sqlite database onto a
> given privacy-oriented contacts database of your own choosing.
>

This is part of the relevant telephony standards, this is a basic how it
does it on GSM, ignoring ACKs and errors and stuff.

Tower sends a RING packet.
Tower sends a (possibly empty) "Calling Number Identification" packet.
Tower sends RING packets until:
- answered - tower sends and receives voice packets
- re-directed - tower sends and receives voice packets
- cancelled - call is disconnected
- times out - call is disconnected
Etcetera.

That's it. To get a look at the "Calling Number Identification" packet
you have to look inside the "call" application.

The actual layout of a "Calling Number Identification" packet is beyond
the scope of this newsfroup, a deep-dive into the source of the Android
"call" application would be required.

Cheers,
Gary B-)

--
Waiting for a new signature to suggest itself...

[JJ]

On Sun, 31 May 2020 19:39:44 -0000 (UTC), Arlen Holder wrote:
> Tough question... likely only for Android freeware experts...
>
> How does incoming caller ID work and more specifically
> is there a method you know of to CONTROL its source data?
>
> Essentially, the goal is to shift incoming CallerID off of the
> system internal default contacts sqlite database onto a
> given privacy-oriented contacts database of your own choosing.

Telephone caller ID source data came from the telephone service (the
telephone company's machine), not Android OS or the recipient's phone
device. So, whether it can be controlled or not, depends on whether the
telephone service provide a way to do that.

[Arlen Holder]

On Mon, 1 Jun 2020 15:31:11 +1000, Gary R. Schmidt wrote:

> This is part of the relevant telephony standards, this is a basic how it
> does it on GSM, ignoring ACKs and errors and stuff.

Hi Gary,

I appreciate that you provided expert information

Thanks for reminding me there are multiple layers of mobile caller ID:
a. The incoming carrier signal "Calling Number Identification" packet
b. The default "Phone app" lookup into the system contacts sqlite db

Hence, "control" of the displayed caller ID has at least two levels.
A. The default "Phone app" recognizing the carrier CNI (if any), and,
B. The default "Phone app" reading the sqlite contacts database (if any).

My goal in understanding is so that I can control the result, such that I
can point the default "Phone app" CID lookup to a desired _local_ database.
--
This thread is an offshoot of the sqlilte caller id problems outlined here:
o Does anyone know how the PHONE ties to CONTACTS tiies to SMS on Android 9
<https://groups.google.com/d/msg/comp.mobile.android/EvXtsP9radE/B1wg5WhfAQAJ>

[Arlen Holder]

On Mon, 1 Jun 2020 19:00:00 +0700, JJ wrote:

> Telephone caller ID source data came from the telephone service (the
> telephone company's machine), not Android OS or the recipient's phone
> device. So, whether it can be controlled or not, depends on whether the
> telephone service provide a way to do that.

Hi JJ,

Thanks for taking a risk at helping answer the technical question, where
you and Gary accurately pointed out that the caller ID information comes
from a _variety_ of cascaded sources.
1. Carrier "Calling Number Identification" packet information
2. Default "Phone app" lookup into the system sqlite contacts database

My goal is to control the _final_ caller ID display, based on a contacts
database of my choosing, on the local device (since I do not have a default
sqlite contacts database, simply for privacy reasons alone).

Googling, there's yet another expansion level of caller ID cascading
o Apparently "caller ID" apps can _also play a role in this cascade.

Given there are _multiple_ cascaded levels to caller id display, the trick
now is to figure out how caller ID REALLY works since it's a cascade of at
least four possible events.
1. carrier CNI information
2. sqlite db contacts lookup
3. caller id app interception
4. personal privacy database lookup

Normally, to find the best free apps, I run a google search
o Where I try to ascertain the overlap of the best in various lists.

First hit is usually the overall survey of what's available:
o *8 Best Caller ID Apps for Android to Identify Incoming Numbers.*
<https://mashtips.com/android-caller-id-app/>
1. Truecaller
<https://play.google.com/store/apps/details?id=com.truecaller>
2. Whoscall
<https://play.google.com/store/apps/details?id=gogolook.callgogolook2>
3. CallApp
<https://play.google.com/store/apps/details?id=com.callapp.contacts>
4. Calls Blacklist
<https://play.google.com/store/apps/details?id=com.vladlee.easyblacklist>
5. Should I Answer
<https://play.google.com/store/apps/details?id=org.mistergroup.muzutozvednout>
6. Mr. Number
<https://play.google.com/store/apps/details?id=com.mrnumber.blocker>
7. Hiya
<https://play.google.com/store/apps/details?id=com.webascender.callerid>
8. CIA
<https://play.google.com/store/apps/details?id=com.ciamedia.caller.id>
9. Showcaller
<https://play.google.com/store/apps/details?id=com.allinone.callerid>

Second hit is the first test of overlap agreement:
o *The Best Caller ID Apps for Android to Screen Your Calls*
<https://joyofandroid.com/best-caller-id-apps-for-android/>
1. Caller ID - Spam Blocker, Phone Dialer & Contacts, by Sync.ME Caller ID
<https://play.google.com/store/apps/details?id=me.sync.callerid>
2. Hiya - Call Blocker, Fraud Detection & Caller ID, by Hiya
<https://play.google.com/store/apps/details?id=com.webascender.callerid>
3. Truecaller: Caller ID, block fraud & scam calls, by True Software
<https://play.google.com/store/apps/details?id=com.truecaller>

Third hit begins to confirm any overlap agreement:
o *5 Best Caller ID Apps For Android, Try It!*
1. Mr. Number - Caller ID & Spam Protection, by Hiya
<https://play.google.com/store/apps/details?id=com.mrnumber.blocker>
2. CallApp: Caller ID, Call Blocker & Call Recorder, by CallApp Caller ID
<https://play.google.com/store/apps/details?id=com.callapp.contacts>
3. Showcaller: Caller ID, Call Recorder & Blocker, by Showcaller
<https://play.google.com/store/apps/details?id=com.allinone.callerid>
4. Truecaller: Caller ID, block fraud & scam calls, by True Software
<https://play.google.com/store/apps/details?id=com.truecaller>
5. Hiya - Call Blocker, Fraud Detection & Caller ID, by Hiya
<https://play.google.com/store/apps/details?id=com.webascender.callerid>

Fourth hit seeks mainly to find new apps, if any:
o *10 Best FREE Caller Id Apps for Android Devices*
<https://www.techykeeday.com/10-best-free-caller-id-apps-for-android-devices/>
#1. TrueCaller
<https://play.google.com/store/apps/details?id=com.truecaller>
#2. Number
#3. HD Full-Screen Caller ID
<https://play.google.com/store/apps/details?id=com.androminigsm.fscifree>
#4. Whoscall
<https://play.google.com/store/apps/details?id=gogolook.callgogolook2>
#5. Hiya
<https://play.google.com/store/apps/details?id=com.webascender.callerid>
#6. CIA
<https://play.google.com/store/apps/details?id=com.ciamedia.caller.id>
#7. TalkSide
<https://talkside.en.aptoide.com/>
#8. Contactive
<https://contactive.en.uptodown.com/android>
#9. Showcaller
<https://play.google.com/store/apps/details?id=com.allinone.callerid>
#10. CallApp
<https://play.google.com/store/apps/details?id=com.callapp.contacts>

Fifth hit starts to have declining relevance:
o *10 Best Caller ID Apps for Android in 2020*
<https://alltipsfinder.com/best-caller-id-apps-android/>
1. Truecaller
2. CIA
3. Hiya
4. Caller ID by CallApp
5. Whoscall
6. Mr. Number
7. Talking Caller ID
8. Showcaller
9. Calls Blacklist
10. Full-Screen Caller ID

There were tons of similar hits, where, luckily, there does seem to be some
overlap, but unluckily, there were very few reviews from sites that I know
from past experience to be reputable freeware review sites.
o *10 Best FREE Caller Id Apps for Android Devices*
<https://www.mobipicker.com/best-caller-id-app-for-android/>

o *Best 10 Caller ID Apps*
<https://appgrooves.com/rank/communication/caller-id/best-caller-id-apps>

o *Top 10 Best Free Caller ID Apps For Android*
<https://andytips.org/top-best-free-caller-id-apps-android/>
etc.

The trick now is to figure out how caller ID REALLY works since it's
apparently an intertwined cascade of at least four possible events.
1. Carrier information
2. sqlite db contacts lookup
3. caller id app interception
4. personal privacy database lookup

[Alan Baker]

On 2020-06-01 10:55 a.m., Arlen Holder wrote:
> On Mon, 1 Jun 2020 19:00:00 +0700, JJ wrote:
>
>> Telephone caller ID source data came from the telephone service (the
>> telephone company's machine), not Android OS or the recipient's phone
>> device. So, whether it can be controlled or not, depends on whether the
>> telephone service provide a way to do that.
>
> Hi JJ,
>
> Thanks for taking a risk at helping answer the technical question, where
> you and Gary accurately pointed out that the caller ID information comes
> from a _variety_ of cascaded sources.
> 1. Carrier "Calling Number Identification" packet information
> 2. Default "Phone app" lookup into the system sqlite contacts database
>
> My goal is to control the _final_ caller ID display, based on a contacts
> database of my choosing, on the local device (since I do not have a default
> sqlite contacts database, simply for privacy reasons alone).

Because sqlite databases on your phone aren't private, Arlen?

[JJ]

On Mon, 1 Jun 2020 11:44:45 -0700, Alan Baker wrote:
>
> Because sqlite databases on your phone aren't private, Arlen?

It's not, if one has something to hide.

[Arlen Holder]

Hi JJ,

As for "something to hide", what on earth are you intimating?

[JJ]

Basic concept of privacy. What were you thinking, anyway?

[Arlen Holder]

On Thu, 4 Jun 2020 00:00:12 +0700, JJ wrote:

>> As for "something to hide", what on earth are you intimating?
>
> Basic concept of privacy. What were you thinking, anyway?

Hi JJ,

You don't seem to be a troll so I will spend energy answering you.

I'm trying to accomplish something _difficult_ here, which is to design a
system _outside_ of google, that does two very important things:
1. It keeps the sqlite file empty (because of known privacy flaws), and,
2. Yet, it allows for caller ID to work for the users' known contacts.

You are either aware of the known sqlite privacy flaws, or you're not.
o I was trying to ascertain, from you, whether you are aware of them.

I _still_ can't tell from your answer whether you're aware of them.
o Are you?

Note: It doesn't matter to the question whether you're aware of them; but
you were responding to a known ignorant troll with the line:

"> Because sqlite databases on your phone aren't private, Arlen?
"It's not, if one has something to hide."

Usenet is useful for purposefully helpful adults...
o That's two things: (a) purposefully helpful, and (b) adults.

You were responding to a troll who is not an adult, which isn't important
because that troll is clearly ignorant of the sqlite privacy holes.

I don't care about that ignorant troll... but I do care about you.

Assuming you're (a) purposefully helpful, and (b) an adult...
o I simply ask you a single question which is relevant to your post:

Q: Are you aware of the known sqlite privacy holes, JJ?

NOTE: If you are, then why did you respond the way you did to the troll?
If not, then all I ask is you try to _understand_ them; that's all.

And _then_, we can get back to the TOPIC of this thread, which is:
Q: How does incoming caller ID work - and more specifically - can caller ID
source data be controlled?

Note this is a TECHNICAL question which affects multiple cascaded levels:

1. carrier CNI information
2. sqlite db contacts lookup
3. caller id app interception
4. personal privacy database lookup

The goal is to freely attain caller ID display _without_ #2 above.
o That's a technical question which the trolls can't possibly help with.
--
Usenet is a medium which requires a bit of clarification in each post.

[Arlen Holder]

On Wed, 3 Jun 2020 17:28:59 -0000 (UTC), Arlen Holder wrote:

> Note this is a TECHNICAL question which affects multiple cascaded levels:
> 1. carrier CNI information
> 2. sqlite db contacts lookup
> 3. caller id app interception
> 4. personal privacy database lookup
>
> The goal is to freely attain caller ID display _without_ #2 above.
> o That's a technical question which the trolls can't possibly help with.

To clarify... I'm not playing silly games like the trolls always play...
o I'm trying to solve a difficult technical problem set - with your help.

Since the trolls love to play silly games, I must repeat the obvious:
o This question is (a) technical, &, inherently (b) difficult to solve.

The reason it's difficult to solve is the solution requires removal of
Google, which itself, requires emptying of the sqlite contacts database.

And yet, the sqlite contacts database is integral to _much_ of Android
o It's integral to how SMS apps & Phone apps _find_ their contacts, and,
o It's integral to how SMS apps & Phone apps _display_ their contacts.

Hence, the problem is (a) technical, and inherently (b) difficult to solve.
o The worthless trolls will have _zero_ chance of solving this problem.

Zero.

If it was as easy as "pay your carrier" for contact information, then I
wouldn't bother asking the question, because anyone can pay their way out
of problems that they don't, themselves, understand.

I'm not interested in playing games with worthless trolls...
o I'm interested in solving this _difficult_ and _technical_ problem.

There , so far, a four-tier cascade to get free caller ID display:
1. carrier CNI information (mine is not free so that's out of the picture)
2. sqlite db contacts lookup (this has known google-based privacy flaws)
3. caller id app interception (these do appear to be free & available)
4. personal privacy database lookup (this, of course, is the best option)

Note that I'm currently working on solving the problem in step #3 or #4.
o That's where I (a)

[JJ]

On Wed, 3 Jun 2020 17:28:59 -0000 (UTC), Arlen Holder wrote:
>
> I _still_ can't tell from your answer whether you're aware of them.
> o Are you?
>
> Note: It doesn't matter to the question whether you're aware of them; but
> you were responding to a known ignorant troll with the line:
> "> Because sqlite databases on your phone aren't private, Arlen?
> "It's not, if one has something to hide."
>
> Usenet is useful for purposefully helpful adults...
> o That's two things: (a) purposefully helpful, and (b) adults.
>
> You were responding to a troll who is not an adult, which isn't important
> because that troll is clearly ignorant of the sqlite privacy holes.
>
> I don't care about that ignorant troll... but I do care about you.
>
> Assuming you're (a) purposefully helpful, and (b) an adult...
> o I simply ask you a single question which is relevant to your post:
>
> Q: Are you aware of the known sqlite privacy holes, JJ?
>
> NOTE: If you are, then why did you respond the way you did to the troll?
> If not, then all I ask is you try to _understand_ them; that's all.

Yes, I do aware of privacy problem of SQLite database. It's the same problem
most other database formats have. But for SQLite, the term "privacy/security
hole" is not applicable, because it doesn't even have any (native) security
feature in the first place.

I don't label non adults as trolls, because even adults can be trolls too.
In fact, adults can be worse. What I do label are trash of society - no
matter how old they are, what gender are they, or what rank/occupation they
have. Those who never contribute to the community and don't care about their
surroundings. I do believe that people are mostly good in nature, so with
caution, I put trust on people first, even though I don't yet know enough
their behavior.

> And _then_, we can get back to the TOPIC of this thread, which is:
> Q: How does incoming caller ID work - and more specifically - can caller ID
> source data be controlled?
>
> Note this is a TECHNICAL question which affects multiple cascaded levels:
> 1. carrier CNI information
> 2. sqlite db contacts lookup
> 3. caller id app interception
> 4. personal privacy database lookup
>
> The goal is to freely attain caller ID display _without_ #2 above.
> o That's a technical question which the trolls can't possibly help with.

First of all, you should know that SQLite database doesn't support native
encryption. Yes, an SQLite can have encrypted data, but _only_ if the SQLite
database engine is custom built with encryption in mind.

What I know is that, In Android, contact list is owned and managed by the
Contacts Storage (system) application. If you want a secure contact list,
that application will need to be replaced.

As for caller ID, that is to altering the caller ID data, or even removing
the caller ID from the remote call event; outside of the system components,
everything is handled by a telephony system application which I'm not yet
sure exactly which one. But whichever that is, it needs to be replaced.

Third party applications, whether it's installed as system applications or
not, can not simply "plugs in" and intercept core events. That's a big
security hole. The only way to "intercept" the events is to replace the
applications which receive the events themselves.

It's a similar case to why there is no third party application that can
filter/block SMS or Class 1 SMS (i.e. popup SMS) messages while preserving
the current (default) SMS application. Mainly because [1] the current
Android architecture doen't provide the needed interception mechanism; and
[2] the current (default) SMS application doesn't provide any plugin API for
third party applications. For caller ID, neither the Telephony or the Phone
applications provide any plugin API for third party applications.

[Arlen Holder]

On Fri, 5 Jun 2020 12:23:46 +0700, JJ wrote:

> Yes, I do aware of privacy problem of SQLite database. It's the same problem
> most other database formats have. But for SQLite, the term "privacy/security
> hole" is not applicable, because it doesn't even have any (native) security
> feature in the first place.

Hi JJ,

Much appreciated your clarification as I'm well aware of the dangers of
trying to help someone on Usenet, where Usenet takes particular patience
since we have to infer "intent" with "sublety" from the written word.

Hence I thank you for patiently explaining that you know about the sqlite
db privacy flaws as my goal, always, is to create a general purpose
solution (by definition, which is free) that anyone can use, which is why
I've written, oh, I can't count them, many tutorials on the net so that
others may follow.

I'm likely the only one you've ever met who has an empty sqlite db. :)
o But, there _are_ apps that understand the need (e.g., Simple Contacts).

So I'm not the only one out there who wants his contacts to remain private.

Since I'm always about helping others in everything that I do...
o I'm hoping, in the end, to have a tutorial for privacy based contacts.

As long as _you_ are aware that there _are_ privacy issues with the default
sqlite database, we don't have to waste any more time on that question.

While you brought up _other_ privacy issues with the default contacts
sqlite db, the key privacy issue I see, is that the instant you use a
Google App, such as GMail, it _insists_ on uploading that sqlite contacts
database, whether you like it or not. From my tests, it doesn't matter
_what_ you want or what you set up; it will do it anyway (I have a thread
on that somewhere).

Given I don't test all Google Apps (e.g., the Google messenger & Dialer and
Google Voice, and Hangouts, etc., I just can't _trust_ any Google app once
I found that out, by experience) - so I don't use Google apps - but I'm
still trying to solve the general purpose issue for everyone - where my
solution has to take into account _other_ people use Google apps.

So the _only_ private sqlite contacts database is an _empty_ one, IMHO.

That should be fine on Android since it's "just" a file; but it's a file
that some SMS/Phone/Contact/Mail apps default to, which is why I choose and
set up my apps to not use that default sqlite contacts database.

I pretty much have it all working fine except that the caller ID display
isn't coming through, but I haven't tested yet the Internet-based caller-id
programs, where my hope is to find one that allows me to manage its own
on-the-phone lookup table.

The fact is that these caller-id apps, if they work, must "intercept" the
caller ID information somewhere in this four-step chain:
1. First the carrier supplies a caller ID (mine doesn't do that for free)
2. Then the default sqlite contacts database supplies a caller id
3. Then, I think, the caller-id apps can intercept this caller-id display
4. And, lastly, I hope to find a "dialer" that has its own caller id db.

> First of all, you should know that SQLite database doesn't support native
> encryption. Yes, an SQLite can have encrypted data, but _only_ if the SQLite
> database engine is custom built with encryption in mind.

This is good to know, as I'm always seeking a general-purpose privacy-based
solution that is freely available, where, in the future, instead of
designing a system with an "empty" sqlite db, maybe an encrypted one might
be better - where the goal is, mostly, to hide from Google apps
automatically grabbing this information and uploading it to the net.

To be clear, I think it should be a criminal offense, punishable by death,
whenever someone uploads my contact information, and that of my kids and
their kids, to the net, without asking for my permission (which I would
never give them).

I think as a substitute for a simple death by hanging on the yardarms,
drawing and quartering would be appropriate for this offense of uploading
_my_ private contact data to the Internet without my permission.

If they simply promise to empty out their sqlite database, then I would
likely petition the governor to grant them a reprieve. :)

> What I know is that, In Android, contact list is owned and managed by the
> Contacts Storage (system) application. If you want a secure contact list,
> that application will need to be replaced.

I think that's a good solution, if I can find it, but in the meantime,
there's a workaround of simply having an _empty_ sqlite contacts database.

> As for caller ID, that is to altering the caller ID data, or even removing
> the caller ID from the remote call event; outside of the system components,
> everything is handled by a telephony system application which I'm not yet
> sure exactly which one. But whichever that is, it needs to be replaced.

This is understood, that "some" telephony application is "intercepting" the
caller-id information from one or more of four currently identified
locations:
1. The cid information from the carrier (mine is empty)
2. The cid information in the contacts sqlite db (mine is empty)
3. The cid information inherent in these cid-display apps (untested by me)
4. The cid information in the "telephony" app itself (untested by me).

I'll attempt to solve #3 but I'm working on a ton of other issues, e.g.,
o *Is there freeware extent to convert Win10S to Win10H WITHOUT enabling the Win10S laptop Wi-Fi?*
<https://groups.google.com/forum/#!topic/alt.comp.freeware/MwzjUei-0Oo>

> Third party applications, whether it's installed as system applications or
> not, can not simply "plugs in" and intercept core events. That's a big
> security hole. The only way to "intercept" the events is to replace the
> applications which receive the events themselves.

This is good to keep in mind, where I have no problem with an empty sqlite
contacts database in getting PulseSMS to identify contacts (after the
fact), and I have no problem searching a non-sqlite contacts vcard file to
initiate calls using SimpleMobileTools' SimpleContacts and/or SimpleDialer.

Given I almost never fail, and yet, the _only_ major problem I haven't
resolved yet is the caller-id display, that's why I'm focusing on how
caller-id works.

Once I better understand how it works, I can better come up with a general
purpose freeware solution that everyone can then benefit from.

> It's a similar case to why there is no third party application that can
> filter/block SMS or Class 1 SMS (i.e. popup SMS) messages while preserving
> the current (default) SMS application. Mainly because [1] the current
> Android architecture doen't provide the needed interception mechanism; and
> [2] the current (default) SMS application doesn't provide any plugin API for
> third party applications. For caller ID, neither the Telephony or the Phone
> applications provide any plugin API for third party applications.

This is also good to keep in mind, where it helps me understand the
workaround to the problem set of caller id given an empty contacts sqlite
database.

Here's a summary of where I stand in writing that general purpose tutorial:
1. There are privacy flaws with the default contacts sqlite database
2. So my easy workaround starts with an _empty_ sqlite contacts database
3. The contacts & dialer can load their own personal vcard contacts file
4. The SMS/MMS app can be tricked to identify all existing contacts
5. That allows easy outgoing contacts search for SMS/MMS and dialing out
5. The only remaining issue is easy caller ID display of known contacts
--
Together, on Usenet, we've written so many useful tutorials it's not funny.

[Gary R. Schmidt]

On 06/06/2020 03:35, Arlen Holder wrote:
[SNIP]

> Hence I thank you for patiently explaining that you know about the sqlite
> db privacy flaws as my goal, always, is to create a general purpose

Please stop saying that SQLite has a "privacy flaw."

It does not.

It relies on other mechanisms to control access.

Please try to be accurate about things. Saying that an internal door
without a lock is insecure when it relies on the external door(s) being
lockable only makes you look foolish.
And, alas, in these days of extreme FUD and bullshit by all and sundry,
may have unfortunate on-going effects.

[Arlen Holder]

On Sat, 6 Jun 2020 13:07:12 +1000, Gary R. Schmidt wrote:

> Please try to be accurate about things. Saying that an internal door
> without a lock is insecure when it relies on the external door(s) being
> lockable only makes you look foolish.

Hi Gary,

Thanks for that purposefully helpful advice...
o Where I'm all about enacting a general purpose working solution.

I don't disagree with you on being more precise about the problem set.
o If you have a _better_ working solution to the problem set, I'm all ears.

Even so, yours is good advice, particularly since we're trying to figure
out how to have an empty default sqlite contacts database and _still_ get
caller id display.

While one possible approach to the problem is to (somehow) "protect" the
default sqlite contacts database, the approach I'm first taking is to
simply eliminate that default sqlite contacts database.

If you, or anyone else, knows how to "protect" that default sqlite contacts
database, I'm all ears, as that would work even better than having an empty
default sqlite contacts database, simply because a _lot_ of important
programs _expect_ the default sqlite contacts database to not be empty.
a. The "Phone app" uses it, by default, to display known contact caller id
b. The "SMS app" uses it, by default, to display known contact caller id
c. Even the "Dialer app" & "Contacts app" use it, by default.

Luckily, I've solved the lack-of-protection problem simply by wiping out
the default sqlite contacts database, and then installing non-Google apps
that don't use that default sqlite contacts database...

But if you have a _better_ working solution to the problem, I'm all ears.
--
It takes effort to undo the fact that MARKETING herds us to slaughter.

[Gary R. Schmidt]

Sure, you have to write a "call application" that does all that you want
securely, it cannot be done using the existing components of an Android
handset.

[Arlen Holder]

On Sat, 6 Jun 2020 22:30:21 +1000, Gary R. Schmidt wrote:

>> But if you have a _better_ working solution to the problem, I'm all ears.
>>
> Sure, you have to write a "call application" that does all that you want
> securely, it cannot be done using the existing components of an Android
> handset.

Hi Gary,

Thank you for clarifying the situation, where I don't disagree with you.
o Both you & I are well meaning, intelligent, purposefully helpful adults.

As such, you hit the nail on the head.
o If you're an expert, you can write your own apps that do have privacy;
o But if you're a general user, you need to rely on privacy-based apps.

The only workable solution is to either write your own app, or simply use
apps that don't rely on the inherently poorly designed default sqlite
contacts database use model (designed, of course, by Google).

To have _anything_ in the default contacts sqlite db, is to NOT be private!
o I didn't design it that way... Google did.

To my knowledge, nobody has suggested a better solution (e.g., your
solution is fine to write your own app, but it's not workable for the
general user).

For the general user...
o If you value privacy, then you *must* have an _empty_ contacts sqlite db!

Yet, for someone like you, who likely _can_ write your own apps (dialer,
contacts, & sms apps), then, to people like you, the statement that the
default sqlite contacts database is inherently not private doesn't ring
true (but only to people who write their own apps).

But, the fact remains, for the general user, this is a VERY TRUE statement:
o *The default sqlite contacts database is inherently _not_ private!*

Worse, for the general user, this is also, sadly, a very true statement:
o *It is constantly uploaded to the Internet _without_ your permission*

Luckily, for the general user, these two workarounds appear to work well:
1. A workable solution is to wipe out that default contacts sqlite db,
2. And then, use non-Google apps that employ a private contacts database.

I've already explained the apps that _do_ respect your privacy
o That is, you need apps that do _not_ use the default sqlite contacts db!

The only app I have left to define is a privacy-aware caller-ID app.
o All it needs to do is read its own private contacts database.

Or, as you correctly stated, you can write your own app that respects your
privacy better than the Google apps do (contacts, dialers, sms, etc.).
--
The key problem with privacy is that Google doesn't want you to have any.

[Gary R. Schmidt]

On 09/06/2020 01:04, Arlen Holder wrote:
> On Sat, 6 Jun 2020 22:30:21 +1000, Gary R. Schmidt wrote:
>
>>> But if you have a _better_ working solution to the problem, I'm all ears.
>>>
>> Sure, you have to write a "call application" that does all that you want
>> securely, it cannot be done using the existing components of an Android
>> handset.
>
> Hi Gary,
>
> Thank you for clarifying the situation, where I don't disagree with you.
> o Both you & I are well meaning, intelligent, purposefully helpful adults.
>
> As such, you hit the nail on the head.
> o If you're an expert, you can write your own apps that do have privacy;
> o But if you're a general user, you need to rely on privacy-based apps.
>
> The only workable solution is to either write your own app, or simply use
> apps that don't rely on the inherently poorly designed default sqlite
> contacts database use model (designed, of course, by Google).
>
> To have _anything_ in the default contacts sqlite db, is to NOT be private!
> o I didn't design it that way... Google did.
>
> To my knowledge, nobody has suggested a better solution (e.g., your
> solution is fine to write your own app, but it's not workable for the
> general user).
>

Ah, I didn't mean that 'you' qua 'you' would have to write the
application, but that a secure application would have to be written.

Possibly one or more already exists, if I wanted secure[1]
communications I would be looking for such a replacement application, as
I said, it cannot be bolted-on to the existing one.

Cheers,
Gary B-)

1 - Of course, secure in this, or any, context is a wibbly-wobbly concept.

[Arlen Holder]

On Tue, 9 Jun 2020 13:35:53 +1000, Gary R. Schmidt wrote:

> Ah, I didn't mean that 'you' qua 'you' would have to write the
> application, but that a secure application would have to be written.

Hi Gary,

I appreciate that you clarified, as Usenet is inherently an easily confused
media, as we converse, days apart, and only via the textual medium.

Unfortunately, I suspect, something like 99.9% of all contacts on people's
Android phones, are _already_ uploaded to the Internet, whether they know
that or not.

Even _my_ contacts, years ago at least, were uploaded to Google servers!
o But this was before I realized what Google was doing with that sqlite db!

All I'm trying to do is what I've always strived to do, which is:
a. Provide a general purpose (usually free) solution that anyone can use
b. Which gives them two things that I feel everyone on Android deserves
(1) Functionality (e.g., contact lookup & caller ID)
(2) With privacy (i.e., contact information should never be on the net!)

I'm close with a solution...
o Oh so close... as I've solved the contacts, dialer, and SMS app issues.

Privacy-based tools that do _not_ use the default contacts sqlite db:
o Dialer === SimpleMobileTools Dialer (a dialer is actually optional today)
o Contacts === SimpleMobileTools Contacts (this is still essential today)
o SMS === PulseSMS (this can be tricked into displaying the caller ID)

Each of those works just fine with an empty default contacts sqlite db.
o *The only thing left is caller ID* (with an empty contacts sqlite db)*

I need to test caller ID apps, but I've been working other issues, e.g.,
o Microsoft installer error: 1326 Failed to install update.
<https://groups.google.com/forum/#!topic/misc.taxes/43gVJ_MxTn4>

Since I'm a good Usenet citizen (e.g., I search first, I ask, I test, and I
report back), I won't have any useful information for you until I test out
the caller-id apps to see if _they_ will solve the caller ID problem that
remains when you wipe out the default contacts sqlite database for privacy.

> Possibly one or more already exists, if I wanted secure[1]
> communications I would be looking for such a replacement application, as
> I said, it cannot be bolted-on to the existing one.

I agree, there are multiple possible solutions to the problem set of the
default contacts sqlite database being uploaded to the net without your
permission.

What are those possible solutions?
o I don't know them all, but these are two that I will strive to find!
1. Find only apps that use their own private personal contacts database!
2. Figure out how to stop Google uploading the default contacts sqlite db.
--
It's a worthwhile endeavor to maintain functionality & privacy together.

[Arlen Holder]

On Tue, 9 Jun 2020 21:26:29 -0000 (UTC), Arlen Holder wrote:

> Privacy-based tools that do _not_ use the default contacts sqlite db:
> o Dialer === SimpleMobileTools Dialer (a dialer is actually optional today)
> o Contacts === SimpleMobileTools Contacts (this is still essential today)
> o SMS === PulseSMS (this can be tricked into displaying the caller ID)
>
> Each of those works just fine with an empty default contacts sqlite db.
> o *The only thing left is caller ID* (with an empty contacts sqlite db)*

Hi Gary and JJ,

There is something people like you, who know stuff, can help me understand.
Android 7 Nougat: <https://i.postimg.cc/j5pqr17b/phone01.jpg>
Android 9 Pie: <https://i.postimg.cc/vByVz7pF/phone09.jpg>

With respect to finding apps that don't give away our privacy, for example:
o <https://i.postimg.cc/3NY6pCMG/phone03.jpg>

I should clarify that I'm still a bit confused what to "call" these things,
since, on my Android 9, there are only _two_ (not three!) default settings:

o Dialer === SimpleMobileTools Dialer (a dialer is actually optional today)
o Contacts === SimpleMobileTools Contacts (this is still essential today)
o SMS === PulseSMS (this can be tricked into displaying the caller ID)

For example, this lists 5 defaults, only 2 of which related to calls/texts:
o Settings > Apps & notifications > Advanced > Default apps

Mine lists these three (which are not confusing to me):
o Assist & voice input === Screenshot Assistant
o Browser app = Tor Browser
o Home app === Nova Launcher

And then mine lists these two, which _are_ confusing to me:
o Phone app = Dialer (which is the "simplemobiletools" dialer app)
o SMS app = Pulse (which is, IMHO, the best SMS/MMS app)
As shown here:
<https://i.postimg.cc/vZ1fHZWj/phone08.jpg>

Notice sometimes I change the settings, and nothing seems to change really:
o Phone app = Contacts (which is the "simplemobiletools" contacts app)
o SMS app = Pulse (which is, IMHO, the best SMS/MMS app)
As shown here:
<https://i.postimg.cc/vByVz7pF/phone09.jpg>

What confuses me are two ephemeral observations I haven't figured out:
(1) Why isn't there 3 defaults (contacts, dialing, & sms/mms)?
(2) Why doesn't it matter what I set the default "Phone app" to?
--
Sometimes you need another perspective to understand what's not intuitive.

[Arlen Holder]

UPDATE:

I updated my Moto G7 from Android 9 to Android 10, where I'd have to check
my screenshots from the past, but it seems in Android 10 there are two new
"Default apps" settings (that were not in Android 9).
o Those on Android 10... is it worth upgrading from 9 to 10?
What are the pitfalls you've experienced & the benefits?
<https://groups.google.com/forum/#!topic/comp.mobile.android/X65cMyzAn-g>

o Android 10 Settings > Apps & notifications > Default apps >
1. Assist app -> Screenshot Assistant
2. Browser app -> Tor Browser
3. *Call redirecting app* -> None (I think this is new!)
4. *Caller ID & spam app* -> None (I think this is new!)
5. Home app -> Nova Launcher
6. Phone app -> Contacts (i.e., SimpleMobileContacts, not Google Contacts!)
7. SMS app -> Pulse

If others know, offhand, how to take advantage of these two new "defaults",
please let me know, as I'm testing out, for years, a phone that is
different from most Android phones in three privacy-aware ways:

A. I do not have a Google Account set up on the phone
(where you can have a Google Account, but just not set up in the OS)

B. I replaced all Google apps which required a login or which phoned Google
(most of which are very easily replaced with privacy-aware apps)

C. I do not have a default sqlite contacts database in the normal location
(because Google tends to upload that db without your knowledge)

The _only_ thing I was having trouble with was incoming caller ID for
contacts that are in my private contacts database, where maybe these two
new defaults in Android 10 can help solve that problem?

3. *Call redirecting app* -> None (I think this is new!)
4. *Caller ID & spam app* -> None (I think this is new!)

[Arlen Holder]

UPDATE:

This new free caller id for T-Mobile/Sprint customers may help!
<https://i.postimg.cc/MHSTXbX2/scamcalls01.jpg>

Everything below is verbatim...

Subject: Say goodbye to scam calls
We're hanging up on scammers for good.
Now that Sprint and T-Mobile have merged, our network is bigger
and better than ever before, with advanced spam-blocking protection
built into its core.

So customers can be better protected at no extra charge.
This means Caller ID is now free for you!

Download the Scam Shield app to turn on Scam Block and Caller ID
and start saying BUH-BYE to scammers.

Get to know all five layers of Scam Shield protection:
* Advanced network technology
* Scam ID and Scam Block
* Caller ID
* PROXY by DIGITS
* Free yearly number changes

Download the Scam Shield app to turn on free Caller ID
and free Scam Block.

Download on the App Store
<https://apps.apple.com/us/app/id1367276365>

Get it on Google Play
<https://play.google.com/store/apps/details?id=com.tmobile.services.nameid>
--
Qual'g service & capable device req'd. Turning on Scam Block might block
calls you want; disable any time. PROXY: Qual'g service & capable device
req'd. 1 per account; may be cancelled for non-use. In some circumstances,
access to 911 may be limited. See DIGITS Terms of Use for additional 911
information.

This is an automated e-mail. Please do not reply.
If you have questions regarding your order or T‑Mobile service, please
visit my.t-mobile.com.

This email was sent by:
T‑Mobile USA, Inc.
P.O. Box 37380
Albuquerque, NM 87176, USA

T-Mobile, the T logo, Magenta and the magenta color are registered
trademarks of Deutsche Telekom AG. ©2020 T‑Mobile USA, Inc.
| Terms of Use | Terms & Conditions | Return Policy | Privacy Policy

[Arlen Holder]

UPDATE

On Sat, 1 Aug 2020 10:50:48 +0000 (UTC), badgolferman wrote:

>> Download on the App Store
>> <https://apps.apple.com/us/app/id1367276365>
>>
>> Get it on Google Play
>> <https://play.google.com/store/apps/details?id=com.tmobile.services.nameid>
>

> Yes, I installed it. Haven't gotten any scam calls yet. Wish there was a
> way to know what it's blocking though.

Hi The Real Bev,

I have a family plan where I pay for a lot of the kids and grandkids since
the family plan is a decent deal (about $25/month per line with unlimited
US calls & text and 4GB high-speed data) - where I saw something that
intimated it might only work for one line... so I need to test that out.

This is a new app to all of us, where, for privacy reasons, I don't keep
any contacts in the default sqlite database simply because Google uploads
them even when you don't ask Google to do that (there is no real protection
other than to have an empty default sqlite contacts database).

Luckily, I find free, ad free, gsf free apps for contacts and phone which
use their own private contacts database, _outside_ of the highly insecure
default contacts sqlite database that other people use without thinking of
the inherent loss of privacy consequences of uploading your friends,
family, kids, and grandkids' contacts to the Internet.

So it will be interesting if this new free, ad free, GSF dependent T-Mobile
"Scam Shield" app will work, given I don't even have a Google ID on my
phone, and given I've turned off all Google services possible on my phone.

I will install T-Mobile Scam Shield version 4.0.0.3205.3205 on my $100 Moto
G7, which I recently updated to Android 10 (just to test it out) and let
you know how it works out.

Apparently you can't "authenticate" it on Wi-Fi; you must authenticate the
Scam Shield while on Mobile Data.

Apparently the Scam Shield updates on T-Mobile's network every 6 minutes,
according to what the app is telling me in the Scam Shield intro.

1. When installing, it gives you the option to enable free caller id.
2. In Settings > Scam Shield features" is a "proxy" phone number.
3. Also at that location, you can change your number for free yearly.

Of course, they try to foist upon you a "Premium" service by saying "I'm
eligible" for their free for 30 days (I'll stick to the "Basic" plan!)
where the $4/month "Premium" service apparently adds:
a. Manage blocked numbers
b. Send call types to voicemail
{i.e., nuisance, telemarketing, survey, political, charity, prison}
c. Reverse number lookup
d. Voicemails sent to your SMS (via visual voicemail)

But I'll stick with the basic plan as I never pay for any software on a
mobile device, since you can get everything you need for free if you're
smart about it.

I'm testing it out now, where it says "Scam Block is on", and where it says
they'll block known scams for me automatically.

They seem to allow you to create three lists inside the Basic app:
A. A list of favorites (which they will never block)
B. A list of unwanted callers to block (logged in the activity log)
C. A list of numbers to send straight to voice mail

And you can set, inside the app, notifications for:
a. A likely scam call is blocked (basic plan)
b. A caller is blocked (premium plan)
c. A call category is sent to voicemail (premium plan)

It shows what your caller ID is that other people see.

Then there's this "PROXY by Digits" thingey...
"You can get one free PROXY number per account on any of our
Magenta or Essential plans"

That's about it for a one-minute inspection of the app so far.
--
The cost of freeware is in testing and finding those which work best.