wire additional sections reordered for EDNS (and keep raw wire around?)

[Jeremy C. Reed]

I had some wire data collected from tcpdump.

I used dns.message.from_wire() to look at it.

The output became different order than the input.

The EDNS rrset in the Additional section moved from last Additional
section rrset to the first Additional.

The dnspython I am using is not up-to-date. Sorry I didn't check if this
behavior is different now. By the way, thank you for the awesome
software. I have been extensively using it for years.

Is there a way to keep the output identical as the input? Another
example would be having input that didn't use compression pointers so
don't want new wire that is different.

For reference see
https://github.com/rthalley/dnspython/pull/677#issuecomment-950326321
"I'm still open to considering ways to keep the raw wire format
around."

The following is a diff of the input vs. the output with a little
commentary added to see it. Jump down to around line 320:

1 0E
2 62
3 80
4 10
5 00
6 01 QDCOUNT
7 00
8 00 ANCOUNT
9 00
10 05 NSCOUNT
11 00
12 05 ARCOUNT
13 03 3 characters of first label in query: www
14 77
15 77
16 77
17 10
18 66
19 61
20 72
21 73
22 69
23 67
24 68
25 74
26 73
27 65
28 63
29 75
30 72
31 69
32 74
33 79
34 03 3 characters of last label in query: com
35 63
36 6F
37 6D
38 00
39 00
40 01 Question Class IN
41 00
42 01 Question Type A
43 C0 START of AUTH section using compression pointers
44 10
45 00
46 02 NS type
47 00
48 01 IN class
49 00 TTL 127800
50 02
51 A3
52 00
53 00
54 12 RDLENGTH
55 03
56 6E
57 73
58 35
59 0B
60 64
61 6E
62 73
63 6D
64 61
65 64
66 65
67 65
68 61
69 73
70 79
71 C0
72 21
73 C0 next auth record
74 10
75 00
76 02 NS
77 00
78 01 IN
79 00
80 02 TTL
81 A3
82 00
83 00
84 06 RDLENGTH
85 03
86 6E
87 73
88 36
89 C0
90 3A
91 C0 Auth record #3
92 10
93 00
94 02
95 00
96 01
97 00
98 02
99 A3
100 00
101 00
102 06
103 03
104 6E
105 73
106 37
107 C0
108 3A
109 C0 Auth record #4
110 10
111 00
112 2B DS type
113 00
114 01 IN class
115 00
116 01 TTL 86400
117 51
118 80
119 00
120 24 RDLENGTH 26
121 EC RDATA
122 26
123 05
124 02
125 36
126 72
127 C3
128 5C
129 FA
130 8F
131 F1
132 4C
133 9C
134 22
135 3B
136 84
137 27
138 7B
139 D6
140 45
141 C0
142 AF
143 54
144 BA
145 D5
146 79
147 03
148 75
149 FE
150 79
151 71
152 61
153 E4
154 80
155 14
156 79
157 C0 Auth record #5
158 10
159 00
160 2E RRSIG type
161 00
162 01 IN class
163 00
164 01 TTL 86400
165 51
166 80
167 00
168 97 RDLENGTH
169 00
170 2B
171 08
172 02
173 00
174 01
175 51
176 80
177 58
178 3B
179 BE
180 AD
181 58
182 32
183 73
184 C5
185 19
186 04
187 03
188 63
189 6F
190 6D
191 00
192 0D
193 6F
194 F8
195 C0
196 C1
197 38
198 7B
199 DD
200 D0
201 A8
202 BB
203 31
204 FE
205 8C
206 69
207 42
208 A6
209 B2
210 92
211 5B
212 F4
213 89
214 71
215 C3
216 0A
217 9F
218 FE
219 2A
220 1A
221 A7
222 B4
223 43
224 DC
225 84
226 DA
227 AE
228 26
229 EE
230 CD
231 6C
232 69
233 8E
234 96
235 60
236 07
237 6C
238 7E
239 2C
240 F6
241 88
242 5E
243 6B
244 FD
245 07
246 35
247 AE
248 95
249 BE
250 16
251 B1
252 62
253 79
254 84
255 60
256 C5
257 31
258 A4
259 A9
260 03
261 F7
262 43
263 B5
264 81
265 8F
266 2D
267 44
268 19
269 EB
270 98
271 5B
272 CF
273 90
274 47
275 3C
276 94
277 BB
278 C2
279 31
280 75
281 00
282 CA
283 BA
284 E3
285 C2
286 DB
287 60
288 68
289 7F
290 CB
291 B7
292 CE
293 5C
294 4C
295 17
296 CD
297 11
298 34
299 BA
300 CA
301 19
302 F1
303 CA
304 F5
305 9D
306 4D
307 B9
308 27
309 99
310 00
311 18
312 EF
313 69
314 60
315 06
316 F4
317 D7
318 90
319 C2 END of RRSIG RDATA
320 +00 This is where wire data changed -- the EDNS from dnspython
321 +00 label length is 0 . (empty)
322 +29 Type 41 OPT (EDNS)
323 +10 CLASS 4096 EDNS Payload size
324 +00
325 +00
326 +00
327 +80 TTL 32768 which is EXT-RCODE and version, 1st bit set: DO
328 +00
329 +00
330 +00
331 C0 Additional section
332 36
333 00
334 01 Type A
335 00
336 01 IN
337 00
338 02 TTL 172800
339 A3
340 00
341 00
342 04 RDLENGTH
343 D0
344 5E
345 94
346 0D
347 C0 Additional section
348 36
349 00
350 1C
351 00
352 01
353 00
354 02
355 A3
356 00
357 00
358 10 RDLENGTH
359 26
360 00
361 18
362 00
363 00
364 05
365 00
366 00
367 00
368 00
369 00
370 00
371 00
372 00
373 00
374 01
375 C0 Additional #4
376 54
377 00
378 01 A type
379 00
380 01
381 00
382 02
383 A3
384 00
385 00
386 04
387 D0 RDATA for A record. It's glue!
388 50
389 7C
390 0D
391 C0 Additional #5
392 66
393 00
394 01
395 00
396 01
397 00
398 02
399 A3
400 00
401 00
402 04
403 D0
404 50
405 7E
406 0D End of A address RDATA
407 -00 EDNS0 from original pcap
408 -00
409 -29
410 -10
411 -00
412 -00
413 -00
414 -80
415 -00
416 -00
417 -00

[Bob Halley]

Dnspython's message object was not designed to be a Wireshark-like packet dissector.  The to_wire() of something you from_wire()'d will be semantically equivalent, but may not have identical wire format due to rrset construction, TTL minimization, opt placement and compression.

Can you say more about your use case?  E.g. is it enough to stash a copy of the original wire and emit it for your purpose, or are you looking for something more Wireshark-like?  Dnspython certainly has the pieces to write a wire-faithful message format, but it would be a different class I think.

[Jeremy C. Reed]

On Sun, 13 Aug 2023, Bob Halley wrote:

> Dnspython's message object was not designed to be a Wireshark-like packet
> dissector.  The to_wire() of something you from_wire()'d will be
> semantically equivalent, but may not have identical wire format due to rrset
> construction, TTL minimization, opt placement and compression.
> Can you say more about your use case?  E.g. is it enough to stash a copy of
> the original wire and emit it for your purpose, or are you looking for
> something more Wireshark-like?  Dnspython certainly has the pieces to write
> a wire-faithful message format, but it would be a different class I think.

Yes it is enough to save the original wire.

My use cases all involve looking at same DNS message and records using
dnspython PLUS:

1) parse the wire data to understand and show the initial
message for education or testing. I view the data in various formats,
like:

1a) plain hex dump output of entire message

1b) hex + ascii dump with offsets (similar to hexdump -C)

1c) show binary bits for wire headers (like 0 0 0 0 0 0 0 1 1 1 0 1 1 0
1 1 for the TXID)

1d) view compression pointers information like position number, label
length, label itself, compression pointer, compression label, etc. I use
struct to parse for example: (rrttl, rdlength) = struct.unpack('!IH',
thewire[position:position + 6])

1e) view the rdata as Ascii decimal and hex and as raw data (like A
address RDATA: 4\"T)

2) Or show there are no compression pointers. I think dnspython
to_wire() always adds them, if it can, even if the original message did
not use them.

3) be able to know length in bytes of the original message. For example,
I check for payload size: ... if len(response.to_wire()) >
EDNS_PAYLOAD_SIZE ...

4) I save the wire data: f.write(response.to_wire()) to look at it later

I already do all the above using to_wire(). I know I can get a lot of
this information without the original wire data, but I want to use it
too for education and for research.

Keeping the original wire would be useful to me.