consul-kv not working in Windows Environment.

40 views
Skip to first unread message

mike wienold

unread,
Mar 25, 2022, 6:05:28 PM3/25/22
to rqlite
I seem to be having issues trying to get consul to work with rqlite in a windows group. according to the log:

[disco] 2022/03/25 14:43:43 failed to update discovery service with Leader details: Put "http://***.***.***.***:8500/v1/kv/rqlite/leader": dial tcp ***.***.***.***:8500: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.


I can manually curl from the command line to that address just fine. 
curl -X PUT http://***.***.***.***:8500/v1/kv/rqlite/leader -d '***.***.***.***'
true

my launch command is (Powershell):
.\rqlited.exe -node-id $env:COMPUTERNAME -http-addr "$($ipv4):4001" -raft-addr "$($ipv4):4009" -raft-election-timeout 10s -disco-mode consul-kv -disco-config '{\"address\": \"http://***.***.***.***:8500\"}' data

I have tried with and without the http in the command but it gives the same result. 
Sadly I don't have the option to use linux in this particular environment. Is this using an internal method or is it using the system Curl? Could this be a config issue with Consul itself?

Philip O'Toole

unread,
Mar 25, 2022, 7:11:55 PM3/25/22
to rql...@googlegroups.com
Hmm, not immediately obvious what's going on. I do test Consul support with a real Consul system in my testing -- but only on Linux. 


I run a Consul container and ensure clustering takes place OK.

Is there any difference between where rqlited is running, and where you run that curl command? Is it from the same machine? I'd like to see the output of /status after this happens.


--
You received this message because you are subscribed to the Google Groups "rqlite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rqlite+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rqlite/07c288f8-f28b-468d-91bc-57e709b5e6e6n%40googlegroups.com.

mike wienold

unread,
Mar 28, 2022, 10:38:35 AM3/28/22
to rqlite
I have run it from both locations. Interestingly in this project Consul is running on the same machine so I am concerned if there is any conflicts I should avoid. 

I don't see any output differences. My only thought since I am launching this from a powershell script with nssm as a service is that it's trying to use Invoke-WebRequest instead of Curl. I added a function to remove the alias but I don't have any other ideas at this point. 

From the Host machine:
C:\VSM\Scripts> curl ***.***.***.227:4001/status?pretty
{
    "build": {
        "branch": "master",
        "build_time": "unknown",
        "commit": "ca82e8cf59515f5b14f22c5b5fec84206b4bb3b9",
        "compiler": "gc",
        "version": "7"
    },
    "cluster": {
        "addr": " ***.***.***.227  :4009",
        "api_addr": " ***.***.***.227  :4001",
        "https": "false"
    },
    "disco": {
        "last_contact": "2022-03-28T07:17:51.2158055-07:00",
        "mode": "consul-kv",
        "register_interval": 3000000000,
        "report_interval": 30000000000
    },
    "http": {
        "auth": "disabled",
        "bind_addr": " ***.***.***.227  :4001",
        "cluster": {
            "local_node_addr": " ***.***.***.227  :4009",
            "timeout": 30000000000
        }
    },
    "node": {
        "current_time": "2022-03-28T07:20:41.5577905-07:00",
        "start_time": "2022-03-25T15:42:25.9681567-07:00",
        "uptime": "63h38m11.3852932s"
    },
    "os": {
        "executable": "C:\\VSM\\Scripts\\Scheduled\\rqlite-latest-win64\\rqlited.exe",
        "hostname": "3DAPPSHIM",
        "page_size": 4096,
        "pid": 5472,
        "ppid": 6592
    },
    "runtime": {
        "GOARCH": "amd64",
        "GOMAXPROCS": 1,
        "GOOS": "windows",
        "num_cpu": 1,
        "num_goroutine": 16,
        "version": "go1.17.7"
    },
    "store": {
        "addr": " ***.***.***.227  :4009",
        "apply_timeout": "10s",
        "db_applied_index": 0,
        "db_conf": {
            "memory": true,
            "fk_constraints": false
        },
        "dir": "data",
        "dir_size": 65536,
        "election_timeout": "10s",
        "fsm_index": 4,
        "heartbeat_timeout": "1s",
        "leader": {
            "addr": " ***.***.***.227  :4009",
            "node_id": ""
        },
        "no_freelist_sync": false,
        "node_id": "3DAPPSHIM",
        "nodes": [
            {
                "id": "3DAPPSHIM",
                "addr": ":4009",
                "suffrage": "Voter"
            }
        ],
        "observer": {
            "dropped": 0,
            "observed": 1
        },
        "raft": {
            "applied_index": 13,
            "bolt": {
                "FreePageN": 0,
                "PendingPageN": 3,
                "FreeAlloc": 12288,
                "FreelistInuse": 40,
                "TxN": 50830,
                "OpenTxN": 0,
                "TxStats": {
                    "PageCount": 12,
                    "PageAlloc": 49152,
                    "CursorCount": 101679,
                    "NodeCount": 10,
                    "NodeDeref": 2,
                    "Rebalance": 0,
                    "RebalanceTime": 0,
                    "Split": 0,
                    "Spill": 6,
                    "SpillTime": 0,
                    "Write": 18,
                    "WriteTime": 29940800
                }
            },
            "commit_index": 13,
            "fsm_pending": 0,
            "last_contact": 0,
            "last_log_index": 13,
            "last_log_term": 11,
            "last_snapshot_index": 0,
            "last_snapshot_term": 0,
            "latest_configuration": "[{Suffrage:Voter ID:3DAPPSHIM Address::4009}]",
            "latest_configuration_index": 0,
            "log_size": 65536,
            "num_peers": 0,
            "protocol_version": 3,
            "protocol_version_max": 3,
            "protocol_version_min": 0,
            "snapshot_version_max": 1,
            "snapshot_version_min": 0,
            "state": "Leader",
            "term": 11
        },
        "request_marshaler": {
            "compression_batch": 5,
            "compression_size": 150,
            "force_compression": false
        },
        "snapshot_interval": 30000000000,
        "snapshot_threshold": 8192,
        "sqlite3": {
            "compile_options": [
                "ATOMIC_INTRINSICS=1",
                "COMPILER=gcc-11.2.0",
                "DEFAULT_AUTOVACUUM",
                "DEFAULT_CACHE_SIZE=-2000",
                "DEFAULT_FILE_FORMAT=4",
                "DEFAULT_JOURNAL_SIZE_LIMIT=-1",
                "DEFAULT_MMAP_SIZE=0",
                "DEFAULT_PAGE_SIZE=4096",
                "DEFAULT_PCACHE_INITSZ=20",
                "DEFAULT_RECURSIVE_TRIGGERS",
                "DEFAULT_SECTOR_SIZE=4096",
                "DEFAULT_SYNCHRONOUS=2",
                "DEFAULT_WAL_AUTOCHECKPOINT=1000",
                "DEFAULT_WAL_SYNCHRONOUS=1",
                "DEFAULT_WORKER_THREADS=0",
                "ENABLE_DBSTAT_VTAB",
                "ENABLE_FTS3",
                "ENABLE_FTS3_PARENTHESIS",
                "ENABLE_RTREE",
                "ENABLE_UPDATE_DELETE_LIMIT",
                "MALLOC_SOFT_LIMIT=1024",
                "MAX_ATTACHED=10",
                "MAX_COLUMN=2000",
                "MAX_COMPOUND_SELECT=500",
                "MAX_DEFAULT_PAGE_SIZE=8192",
                "MAX_EXPR_DEPTH=1000",
                "MAX_FUNCTION_ARG=127",
                "MAX_LENGTH=1000000000",
                "MAX_LIKE_PATTERN_LENGTH=50000",
                "MAX_MMAP_SIZE=0x7fff0000",
                "MAX_PAGE_COUNT=1073741823",
                "MAX_PAGE_SIZE=65536",
                "MAX_SQL_LENGTH=1000000000",
                "MAX_TRIGGER_DEPTH=1000",
                "MAX_VARIABLE_NUMBER=32766",
                "MAX_VDBE_OP=250000000",
                "MAX_WORKER_THREADS=8",
                "MUTEX_W32",
                "OMIT_DEPRECATED",
                "OMIT_SHARED_CACHE",
                "SYSTEM_MALLOC",
                "TEMP_STORE=1",
                "THREADSAFE=1"
            ],
            "conn_pool_stats": {
                "ro": {
                    "max_open_connections": 0,
                    "open_connections": 1,
                    "in_use": 0,
                    "idle": 1,
                    "wait_count": 0,
                    "wait_duration": 0,
                    "max_idle_closed": 0,
                    "max_idle_time_closed": 0,
                    "max_lifetime_closed": 0
                },
                "rw": {
                    "max_open_connections": 1,
                    "open_connections": 1,
                    "in_use": 0,
                    "idle": 1,
                    "wait_count": 0,
                    "wait_duration": 0,
                    "max_idle_closed": 0,
                    "max_idle_time_closed": 0,
                    "max_lifetime_closed": 0
                }
            },
            "db_size": 0,
            "mem_stats": {
                "cache_size": -2000,
                "freelist_count": 0,
                "hard_heap_limit": 0,
                "max_page_count": 1073741823,
                "page_count": 0,
                "page_size": 4096,
                "soft_heap_limit": 0
            },
            "path": ":memory:",
            "ro_dsn": "file:/glrMLmlmGmosNsotmmMo?mode=ro\u0026vfs=memdb\u0026_txlock=deferred\u0026_fk=false",
            "rw_dsn": "file:/glrMLmlmGmosNsotmmMo?mode=rw\u0026vfs=memdb\u0026_txlock=immediate\u0026_fk=false",
            "version": "3.38.0"
        },
        "startup_on_disk": false,
        "trailing_logs": 10240
    }
}


From My Machine:
C:\vsm\Scripts\Scheduled\rqlite-latest-win64> curl  ***.***.***.227  :4001/status?pretty
{
    "build": {
        "branch": "master",
        "build_time": "unknown",
        "commit": "ca82e8cf59515f5b14f22c5b5fec84206b4bb3b9",
        "compiler": "gc",
        "version": "7"
    },
    "cluster": {
        "addr": " ***.***.***.227  :4009",
        "api_addr": " ***.***.***.227  :4001",
        "https": "false"
    },
    "disco": {
        "last_contact": "2022-03-28T07:12:51.216577-07:00",
        "mode": "consul-kv",
        "register_interval": 3000000000,
        "report_interval": 30000000000
    },
    "http": {
        "auth": "disabled",
        "bind_addr": " ***.***.***.227  :4001",
        "cluster": {
            "local_node_addr": " ***.***.***.227  :4009",
            "timeout": 30000000000
        }
    },
    "node": {
        "current_time": "2022-03-28T07:14:46.949677-07:00",
        "start_time": "2022-03-25T15:42:25.9681567-07:00",
        "uptime": "63h32m16.7822946s"
    },
    "os": {
        "executable": "C:\\VSM\\Scripts\\Scheduled\\rqlite-latest-win64\\rqlited.exe",
        "hostname": "3DAPPSHIM",
        "page_size": 4096,
        "pid": 5472,
        "ppid": 6592
    },
    "runtime": {
        "GOARCH": "amd64",
        "GOMAXPROCS": 1,
        "GOOS": "windows",
        "num_cpu": 1,
        "num_goroutine": 16,
        "version": "go1.17.7"
    },
    "store": {
        "addr": " ***.***.***.227  :4009",
        "apply_timeout": "10s",
        "db_applied_index": 0,
        "db_conf": {
            "memory": true,
            "fk_constraints": false
        },
        "dir": "data",
        "dir_size": 65536,
        "election_timeout": "10s",
        "fsm_index": 4,
        "heartbeat_timeout": "1s",
        "leader": {
            "addr": " ***.***.***.227  :4009",
            "node_id": ""
        },
        "no_freelist_sync": false,
        "node_id": "3DAPPSHIM",
        "nodes": [
            {
                "id": "3DAPPSHIM",
                "addr": ":4009",
                "suffrage": "Voter"
            }
        ],
        "observer": {
            "dropped": 0,
            "observed": 1
        },
        "raft": {
            "applied_index": 13,
            "bolt": {
                "FreePageN": 0,
                "PendingPageN": 3,
                "FreeAlloc": 12288,
                "FreelistInuse": 40,
                "TxN": 50749,
                "OpenTxN": 0,
                "TxStats": {
                    "PageCount": 12,
                    "PageAlloc": 49152,
                    "CursorCount": 101517,
                    "NodeCount": 10,
                    "NodeDeref": 2,
                    "Rebalance": 0,
                    "RebalanceTime": 0,
                    "Split": 0,
                    "Spill": 6,
                    "SpillTime": 0,
                    "Write": 18,
                    "WriteTime": 29940800
                }
            },
            "commit_index": 13,
            "fsm_pending": 0,
            "last_contact": 0,
            "last_log_index": 13,
            "last_log_term": 11,
            "last_snapshot_index": 0,
            "last_snapshot_term": 0,
            "latest_configuration": "[{Suffrage:Voter ID:3DAPPSHIM Address::4009}]",
            "latest_configuration_index": 0,
            "log_size": 65536,
            "num_peers": 0,
            "protocol_version": 3,
            "protocol_version_max": 3,
            "protocol_version_min": 0,
            "snapshot_version_max": 1,
            "snapshot_version_min": 0,
            "state": "Leader",
            "term": 11
        },
        "request_marshaler": {
            "compression_batch": 5,
            "compression_size": 150,
            "force_compression": false
        },
        "snapshot_interval": 30000000000,
        "snapshot_threshold": 8192,
        "sqlite3": {
            "compile_options": [
                "ATOMIC_INTRINSICS=1",
                "COMPILER=gcc-11.2.0",
                "DEFAULT_AUTOVACUUM",
                "DEFAULT_CACHE_SIZE=-2000",
                "DEFAULT_FILE_FORMAT=4",
                "DEFAULT_JOURNAL_SIZE_LIMIT=-1",
                "DEFAULT_MMAP_SIZE=0",
                "DEFAULT_PAGE_SIZE=4096",
                "DEFAULT_PCACHE_INITSZ=20",
                "DEFAULT_RECURSIVE_TRIGGERS",
                "DEFAULT_SECTOR_SIZE=4096",
                "DEFAULT_SYNCHRONOUS=2",
                "DEFAULT_WAL_AUTOCHECKPOINT=1000",
                "DEFAULT_WAL_SYNCHRONOUS=1",
                "DEFAULT_WORKER_THREADS=0",
                "ENABLE_DBSTAT_VTAB",
                "ENABLE_FTS3",
                "ENABLE_FTS3_PARENTHESIS",
                "ENABLE_RTREE",
                "ENABLE_UPDATE_DELETE_LIMIT",
                "MALLOC_SOFT_LIMIT=1024",
                "MAX_ATTACHED=10",
                "MAX_COLUMN=2000",
                "MAX_COMPOUND_SELECT=500",
                "MAX_DEFAULT_PAGE_SIZE=8192",
                "MAX_EXPR_DEPTH=1000",
                "MAX_FUNCTION_ARG=127",
                "MAX_LENGTH=1000000000",
                "MAX_LIKE_PATTERN_LENGTH=50000",
                "MAX_MMAP_SIZE=0x7fff0000",
                "MAX_PAGE_COUNT=1073741823",
                "MAX_PAGE_SIZE=65536",
                "MAX_SQL_LENGTH=1000000000",
                "MAX_TRIGGER_DEPTH=1000",
                "MAX_VARIABLE_NUMBER=32766",
                "MAX_VDBE_OP=250000000",
                "MAX_WORKER_THREADS=8",
                "MUTEX_W32",
                "OMIT_DEPRECATED",
                "OMIT_SHARED_CACHE",
                "SYSTEM_MALLOC",
                "TEMP_STORE=1",
                "THREADSAFE=1"
            ],
            "conn_pool_stats": {
                "ro": {
                    "max_open_connections": 0,
                    "open_connections": 1,
                    "in_use": 0,
                    "idle": 1,
                    "wait_count": 0,
                    "wait_duration": 0,
                    "max_idle_closed": 0,
                    "max_idle_time_closed": 0,
                    "max_lifetime_closed": 0
                },
                "rw": {
                    "max_open_connections": 1,
                    "open_connections": 1,
                    "in_use": 0,
                    "idle": 1,
                    "wait_count": 0,
                    "wait_duration": 0,
                    "max_idle_closed": 0,
                    "max_idle_time_closed": 0,
                    "max_lifetime_closed": 0
                }
            },
            "db_size": 0,
            "mem_stats": {
                "cache_size": -2000,
                "freelist_count": 0,
                "hard_heap_limit": 0,
                "max_page_count": 1073741823,
                "page_count": 0,
                "page_size": 4096,
                "soft_heap_limit": 0
            },
            "path": ":memory:",
            "ro_dsn": "file:/glrMLmlmGmosNsotmmMo?mode=ro\u0026vfs=memdb\u0026_txlock=deferred\u0026_fk=false",
            "rw_dsn": "file:/glrMLmlmGmosNsotmmMo?mode=rw\u0026vfs=memdb\u0026_txlock=immediate\u0026_fk=false",
            "version": "3.38.0"
        },
        "startup_on_disk": false,
        "trailing_logs": 10240
    }
}

Philip O'Toole

unread,
Mar 28, 2022, 11:31:08 AM3/28/22
to rql...@googlegroups.com
I think your command line is bad. More inline.

On Fri, Mar 25, 2022 at 6:05 PM mike wienold <drac...@gmail.com> wrote:


my launch command is (Powershell):
.\rqlited.exe -node-id $env:COMPUTERNAME -http-addr "$($ipv4):4001" -raft-addr "$($ipv4):4009" -raft-election-timeout 10s -disco-mode consul-kv -disco-config '{\"address\": \"http://***.***.***.***:8500\"}' data

As per the docs, the Consul address shouldn't include 'http'. I can modify the code to catch this error, but in the meantime you should fix your command line. See the docs below for more details.


mike wienold

unread,
Mar 28, 2022, 11:31:55 AM3/28/22
to rqlite
I should also add: 
curl ***.***.***.227:4001/nodes
{"3DAPPSHIM":{"addr":":4009","reachable":false,"leader":false,"error":"factory is not able to fill the pool: dial tcp :4009: connectex: No connection could be made because the target machine actively refused it."}}
I tried changing the raft port but I don't understand why it's being refused. there is nothing to block (firewall, Router, etc...)

Philip O'Toole

unread,
Mar 28, 2022, 12:05:02 PM3/28/22
to rql...@googlegroups.com
On Mon, Mar 28, 2022 at 11:31 AM mike wienold <drac...@gmail.com> wrote:
I should also add: 
curl ***.***.***.227:4001/nodes
{"3DAPPSHIM":{"addr":":4009","reachable":false,"leader":false,"error":"factory is not able to fill the pool: dial tcp :4009: connectex: No connection could be made because the target machine actively refused it."}}
I tried changing the raft port but I don't understand why it's being refused. there is nothing to block (firewall, Router, etc...)

OK, well, let's focus on one issue at a time. As mentioned above, your rqlite command line looks wrong, the Consul config should not have 'http' in the address. Once we fix that up, we can look into your networking issues.
 

mike wienold

unread,
Mar 28, 2022, 1:15:05 PM3/28/22
to rqlite
ok now I have to laugh at myself as far as Consul goes. I apparently mistyped the address. 228 should have been 227. so that part works now.  so now they are all fighting over who is leader and not talking to each other...

            _ _ _
           | (_) |
  _ __ __ _| |_| |_ ___
 | '__/ _  | | | __/ _ \   The lightweight, distributed
 | | | (_| | | | ||  __/   relational database.
 |_|  \__, |_|_|\__\___|
         | |               www.rqlite.io
         |_|

[rqlited] 2022/03/28 10:08:14 rqlited starting, version 7, commit ca82e8cf59515f5b14f22c5b5fec84206b4bb3b9, branch master, compiler gc
[rqlited] 2022/03/28 10:08:14 go1.17.7, target architecture is amd64, operating system target is windows
[rqlited] 2022/03/28 10:08:14 launch command: C:\VSM\Scripts\Scheduled\rqlite-latest-win64\rqlited.exe -node-id W2C-VOXP103 -http-addr ***.***.***.56:4001 -raft-addr  ***.***.***.56:4009 -raft-election-timeout 10s -disco-mode consul-kv -disco-config {"address": " ***.***.***.227:8500"} data
[rqlited] 2022/03/28 10:08:14 Raft TCP mux Listener registered with 1
[mux] 2022/03/28 10:08:14 mux serving on  ***.***.***.56:4009, advertising  ***.***.***.56:4009
[rqlited] 2022/03/28 10:08:14 no preexisting node state detected in C:\VSM\Scripts\Scheduled\rqlite-latest-win64\data, node may be bootstrapping
[store] 2022/03/28 10:08:14 opening store with node ID W2C-VOXP103
[store] 2022/03/28 10:08:14 configured for an in-memory database
[store] 2022/03/28 10:08:14 ensuring directory for Raft exists at data
[store] 2022/03/28 10:08:14 0 preexisting snapshots present
[store] 2022/03/28 10:08:14 first log index: 0, last log index: 0, last command log index: 0:
[store] 2022/03/28 10:08:14 created in-memory database at open
2022-03-28T10:08:14.654-0700 [INFO]  raft: initial configuration: index=0 servers=[]
[cluster] 2022/03/28 10:08:14 service listening on  ***.***.***.56  :4009
[rqlited] 2022/03/28 10:08:14 cluster TCP mux Listener registered with 2
2022-03-28T10:08:14.654-0700 [INFO]  raft: entering follower state: follower="Node at  ***.***.***.56:4009 [Follower]" leader=
[http] 2022/03/28 10:08:14 service listening on  ***.***.***.56:4001
[rqlited] 2022/03/28 10:08:14 discovery mode: consul-kv
[rqlited] 2022/03/28 10:08:14 no preexisting nodes, registering with discovery service
[rqlited] 2022/03/28 10:08:14 discovery service returned http://***.***.***.15:4001 as join address
2022-03-28T10:08:16.376-0700 [WARN]  raft: no known peers, aborting election
[cluster-join] 2022/03/28 10:08:35 failed to join cluster at [http://***.***.***.15:4001]: Post "http://***.***.***.15:4001/join": dial tcp  ***.***.***.15:4001: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., sleeping 3s before retry

Philip O'Toole

unread,
Mar 28, 2022, 2:05:20 PM3/28/22
to rql...@googlegroups.com
[cluster-join] 2022/03/28 10:08:35 failed to join cluster at [http://***.***.***.15:4001]: Post "http://***.***.***.15:4001/join": dial tcp  ***.***.***.15:4001: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., sleeping 3s before retry

This is the key failure in the log. Consul tells the node where the leader is, but this node can't contact the leader at that address. You'll need to figure out why, but this looks like something your network -- or in the data returned by Consul. I would check Consul, and see what network address are registered in the rqlite key. The join code is well tested.

You could try a manual join first, just to ensure everything is working. Get Consul out of the loop.


In fact maybe you should create your cluster by hand (as per the doc above) to ensure your network is operating correctly between the nodes. Consul is just confusing the situation right now. Then, once you can create a network manually, then introduce Consul.

To view this discussion on the web visit https://groups.google.com/d/msgid/rqlite/45b41ff1-ee48-4d3a-b42a-c7b4efe55f14n%40googlegroups.com.

mike wienold

unread,
Mar 28, 2022, 2:26:28 PM3/28/22
to rqlite
that's the part that confuses me, it doesn't matter what I set the raft port to, since I knew 4001 was working as the http I tried to use that for raft and change the http port to 4000 and that still results in failure. 4000 responds to curl fine but whatever the raft port is It is rejected. 
I had tried this without consul FYI, for whatever reason I thought consul would help eliminate some issues. 

Philip O'Toole

unread,
Mar 28, 2022, 2:37:43 PM3/28/22
to rql...@googlegroups.com
I'd need to see a fuller example of a failing clustering operation, manually performed. What I mean by this is to see a) each of 3 command lines you run, b) the full log from each node after clustering failings, and c) the output of /status from each node. That would help me diagnose what's going wrong.

To view this discussion on the web visit https://groups.google.com/d/msgid/rqlite/6796e710-b084-49c3-a783-d3a5ded0187en%40googlegroups.com.

mike wienold

unread,
Mar 28, 2022, 3:13:24 PM3/28/22
to rqlite
I started completely over and deleted the data directory. removed Consul from the mix, and now it works.... 
launch command: C:\VSM\Scripts\Scheduled\rqlite-latest-win64\rqlited.exe -node-id 3DAPPSHIM -http-addr ***.***.***.227:4000 -raft-addr ***.***.***.227:4001 -raft-election-timeout 100s data
each of the other servers are running something equivalent to this, The host IP addresses are identified via script. 
launch command: C:\VSM\Scripts\Scheduled\rqlite-latest-win64\rqlited.exe -node-id W2C-VOXP109 -http-addr ***.***.***.***:4000 -raft-addr ***.***.***.48:4001 -raft-election-timeout 10s -join http://***.***.***.227:4000 data

I still have a few other nodes that are pulling an old config but those should sync up in 30 minutes or so when they poll for the new config files. I guess the question is should I attempt to add consul back in. will a node that drops off just attempt to contact the -join address or would it try to contact all of it's last known nodes?

Philip O'Toole

unread,
Mar 28, 2022, 3:28:47 PM3/28/22
to rql...@googlegroups.com
On Mon, Mar 28, 2022 at 3:13 PM mike wienold <drac...@gmail.com> wrote:
I started completely over and deleted the data directory. removed Consul from the mix, and now it works.... 
launch command: C:\VSM\Scripts\Scheduled\rqlite-latest-win64\rqlited.exe -node-id 3DAPPSHIM -http-addr ***.***.***.227:4000 -raft-addr ***.***.***.227:4001 -raft-election-timeout 100s data
each of the other servers are running something equivalent to this, The host IP addresses are identified via script. 
launch command: C:\VSM\Scripts\Scheduled\rqlite-latest-win64\rqlited.exe -node-id W2C-VOXP109 -http-addr ***.***.***.***:4000 -raft-addr ***.***.***.48:4001 -raft-election-timeout 10s -join http://***.***.***.227:4000 data

I still have a few other nodes that are pulling an old config but those should sync up in 30 minutes or so when they poll for the new config files. I guess the question is should I attempt to add consul back in. will a node that drops off just attempt to contact the -join address or would it try to contact all of it's last known nodes?

Once a node is in a cluster, any Consul-related options are ignored for the purpose of joining. Consul is to help nodes *find a node through which they can join a cluster). Once it joins a cluster (regardless of how it did that), it keep the records of other nodes in its Raft log (which is stored in the data directory).

Nodes that have never been part of a cluster can use Consul at start up to find the leader of a given cluster however. But again, once they join the cluster, they will never use Consul again to find other nodes in the cluster.

Does that make sense?
 

mike wienold

unread,
Mar 28, 2022, 3:56:07 PM3/28/22
to rqlite
ok, thank you for the confirmation. 
Reply all
Reply to author
Forward
0 new messages