Directory size exceeds file

ider

unread,

Mar 11, 2021, 5:28:48 AM3/11/21

to Seaweed File System

version 2.23

first problem:

Directory use 147GB,but file use only 75GB, where is the space

second problem:

in filer server:

there are many log like

E0311 18:14:35 40379 stream.go:39] read chunk: http://192.168.1.229:17071/426,415aa64f392fbd?readDeleted=true: 404 Not Found,

some file lost in volume 426, but in weed shell volume list output

volume id:426 size:8 collection:"mag" version:3 compact_revision:1 modified_at_second:1615358553

volume id:427 size:8065704 collection:"mag" file_count:2 version:3 modified_at_second:1615354531

volume id:439 size:26226107720 collection:"mag" file_count:3648 delete_count:651 deleted_byte_count:4442393624 version:3 modified_at_second:1615452392

volume id:444 size:26231420544 collection:"mag" file_count:3537 delete_count:269 deleted_byte_count:1907440829 version:3 modified_at_second:1615451880

there maybe a bug, filer send filer to volume 426, but volume 426 lost data, and filer can not get data from volume 426

Chris Lu

unread,

Mar 11, 2021, 12:28:33 PM3/11/21

to Seaweed File System

1. Can not tell. This is not reported by SeaweedFS, but your native file system.

2. Seems volume 426 content was missing somehow. But can not tell more than that.

--
You received this message because you are subscribed to the Google Groups "Seaweed File System" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seaweedfs+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/seaweedfs/0b6e82a4-67df-4c70-aea2-322249bd318an%40googlegroups.com.

ider

unread,

Mar 12, 2021, 4:35:32 AM3/12/21

to Seaweed File System

thanks， i got some answers ，

1. i use volumePreallocate in maser and Disk format is xfs, disappear space is Preallocate for volume but not show in command ls

today , i update weed 2.23 -> 2.31, A new problem has arisen。

When I read files concurrently， in a litter while , filer and volume start output error log

filer log

E0312 16:55:06 78076 stream.go:39] read chunk: read tcp 192.168.1.222:54734->192.168.1.220:17071: i/o timeout

E0312 16:55:06 78076 stream.go:39] read chunk: read tcp 192.168.1.222:54736->192.168.1.220:17071: i/o timeout

E0312 16:55:06 78076 stream.go:39] read chunk: read tcp 192.168.1.222:54738->192.168.1.220:17071: i/o timeout

E0312 16:55:07 78076 stream.go:39] read chunk: read tcp 192.168.1.222:54740->192.168.1.220:17071: i/o timeout

I0312 17:05:50 78076 filechunk_manifest.go:112] read http://192.168.1.220:17071/252,35bcfa7bfee586 failed, err: read tcp 192.168.1.222:55670->192.168.1.220:17071: i/o timeout

I0312 17:05:50 78076 filechunk_manifest.go:119] retry reading in 1s

I0312 17:05:50 78076 filechunk_manifest.go:112] read http:// 192.168.1.220 :17071/174,2ea2053a6ed180 failed, err: read tcp 192.168.1.222:55672->192.168.1.220:17071: i/o timeout

I0312 17:05:50 78076 filechunk_manifest.go:119] retry reading in 1s

I0312 17:05:50 78076 filechunk_manifest.go:112] read http:// 192.168.1.220 :17071/384,3ef9dcb6317a32 failed, err: read tcp 192.168.1.222:55674->192.168.1.220:17071: i/o timeout

volume log

2021/03/12 17:06:59 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:242)

in client , file get fail

i back off filer to v2.23 only, there problem disappear

Chris Lu

unread,

Mar 12, 2021, 1:00:54 PM3/12/21

to Seaweed File System

Need to have a way to reproduce.

Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/seaweedfs/8b579e8e-7ed5-4cda-8cb3-239c304a5486n%40googlegroups.com.

Reply all

Reply to author

Forward