3d Object Reconstruction

[Michael]

Iam trying to figure out how to use ReCap photo for object reconstruction. There doesn't seem to be any information or even a rudimentary guide anywhere in the knowledge network, although since most links seem to be broken I might just not be finding it.

There's probably more, but these are a good starting point I think. The UI/UX of the knowledge base has me ready to pull my hair out. Even if most of the links didn't result in 404 pages, the fact that the "learn" section is just a functionally-random jumble of search results and filters is borderline inexcusable.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Remembering is a reconstructive process, yet little is known about how the reconstruction of a memory unfolds in time in the human brain. Here, we used reaction times and EEG time-series decoding to test the hypothesis that the information flow is reversed when an event is reconstructed from memory, compared to when the same event is initially being perceived. Across three experiments, we found highly consistent evidence supporting such a reversed stream. When seeing an object, low-level perceptual features were discriminated faster behaviourally, and could be decoded from brain activity earlier, than high-level conceptual features. This pattern reversed during associative memory recall, with reaction times and brain activity patterns now indicating that conceptual information was reconstructed more rapidly than perceptual details. Our findings support a neurobiologically plausible model of human memory, suggesting that memory retrieval is a hierarchical, multi-layered process that prioritises semantically meaningful information over perceptual details.

When Rocky Balboa goes back to his old gym in the movie Rocky V, the boxing ring and the feeling of the dusted gloves in his hands trigger a flood of vivid images from the past. Like in many other movies featuring such mnemonic flashbacks, the main character seems capable of remembering what the room looked like years ago, who was there at the time, and even an emotional conversation with his old friend and coach Mickey. Perceptual details like colours, however, are initially missing in the scene, like in a faded photograph, and only gradually saturate over time. This common way of depicting memories in pop culture nicely illustrates that the memories we bring back to mind are not unitary constructs, and also not veridical copies of past events. Instead, it suggests that remembering is a reconstructive process that prioritises more meaningful components of an event over other, more shallow aspects1,2. We here report three experiments that shed light onto the temporal information flow during memory retrieval. Once a reminder has elicited a stored memory trace, are the different features of this memory reconstructed in a systematic, hierarchical way?

Surprisingly little is known about the time course of memory recall, considering our vast knowledge about the information processing hierarchy during visual perception. Visual object recognition is generally assumed to progress from low-level perceptual features, processed in early visual areas, to increasingly higher levels of integration and abstraction along the inferior temporal cortex3,4,5,6,7,8. What if a mental representation is re-created from memory, without much external stimulation? Retrieving a scene from Rocky V will elicit semantic knowledge about the film (e.g. the actor being Sylvester Stallone), but also mental images that can include fairly low-level details (e.g. whether the scene was in colour or in grey scale). How the brain manages to bring back each of these features when reconstructing an event from memory remains an open question. The present series of experiments tested our central working hypothesis that the stream of information processing is reversed during memory reconstruction compared with the perception of an external stimulus.

We hypothesise that such a processing hierarchy does exist, and that the information flow is reversed during memory retrieval compared with perception. That is, based on the widely accepted idea that memory reconstruction depends on back-projections from the hippocampus to neo-cortex21,22, we expect that those areas that are anatomically closer to the hippocampus (i.e., high-level conceptual processing areas along the inferior temporal cortex) are involved in the reactivation cascade faster than relatively remote areas (i.e., low-level perceptual processing areas). Therefore, we assume that once a reminder has initiated the reactivation of an associated event, higher-level abstract features will be reconstructed before lower-level perceptual features, producing an inverse temporal order of processing compared with perception.

We tested this reverse reconstruction hypothesis in a series of two behavioural and one electroencephalography (EEG) experiment. All studies used a simple associative memory paradigm where participants learn arbitrary associations between word cues and everyday objects, and are later cued with the word to recall the object. In order to test for a processing hierarchy, it was important to independently manipulate the perceptual and conceptual contents of these objects. Therefore, objects varied along two orthogonal dimensions: one perceptual dimension, where the object was either presented as a photograph or a line drawing; and a semantic dimension where the object represents an animate or inanimate entity (Fig. 1a). The two behavioural experiments measure reaction times (RTs) while participants make perceptual or semantic category judgments for objects that are either visually presented on the screen, or reconstructed from memory. The EEG experiment uses a similar associative recall paradigm together with time-series decoding techniques3,4,23, allowing us to track at which exact moment in time perceptual and semantic components of the same object are reactivated, and to create a temporal map of semantic and perceptual features during perception and memory reconstruction. Our behavioural and electrophysiological findings consistently support the idea that memory reconstruction is not an all-or-none process, but rather progresses from higher-level semantic to lower-level perceptual features.

Stimuli and design of the behavioural experiments. a Illustration of the orthogonal design of the stimulus set. In all experiments, objects (a total of 128) varied along two dimensions: a perceptual dimension where objects could be presented as a photograph or as a line drawing; and a semantic dimension where objects could belong to the animate or inanimate category. b In the visual reaction time task, participants were prompted on each trial to categorise the upcoming object as fast as possible, either according to its perceptual category (photograph vs. line drawing) or its semantic category (animate vs. inanimate). c During the encoding phase of a memory reaction time task, participants were asked to create word-object associations (a total of eight per block). Reaction times were then measured during the retrieval phase, where subjects were presented with a reminder word, and asked to recall and categorise the associated object according to its perceptual (photograph vs. line drawing) or semantic (animate vs. inanimate) features. Button press symbols indicate at which moment in a trial RTs were collected

Our two behavioural experiments used RTs to test our central hypothesis that the information processing hierarchy reverses between the visual perception of an object and its reconstruction from memory. We assumed that the time required to answer a question about low-level perceptual features (photograph vs. drawing) compared to high-level semantic features (animate vs. inanimate) of an item reflects the speed at which these types of information become available in the brain. If so, reaction time patterns should reverse depending on whether the object is visually presented or reconstructed from memory: during perception, RTs should be faster for perceptual compared with semantic questions reflecting a forward processing hierarchy; during retrieval, RTs should be faster for semantic compared with perceptual questions if there is a reversal of that hierarchy.

To directly test for a reversal of the reaction time pattern between visual perception and memory reconstruction, we used generalised linear mixed-effect models (GLMM). GLMMs are ideal for modelling single trial (e.g. RT) data, without assumptions about the underlying distribution. They are able to capture variance explained by fixed and random variables, including the experimental manipulations of interest24. We used single trial RTs as target (dependent) variable. Fixed effects were the kind of task (visual vs. memory), question type (perceptual vs. semantic) and the interaction between task and question type. Participant IDs and slopes were included as random factors (including intercept).

Reaction time analyses thus support our central hypothesis that the speed of information processing for different object features reverses between perception and memory, a pattern replicated between Experiments 1 and 2.

Next we investigated if a similar pattern was present in terms of accuracy (Fig. 2d, e). We used a GLMM with a logistic link function and a binary probability distribution for our target variable (accuracy, correct or incorrect on a given single trial). Fixed effects were the type of task (visual vs. memory), question type (perceptual vs. semantic), and the interaction between the two factors. Participant IDs and slopes were selected as random factors, including intercept.

Altogether, the findings from our two behavioural experiments support our main hypothesis that during retrieval of a complex visual representation, the temporal order in which perceptual and semantic features are processed reverses compared with the initial perception. The results suggest that RTs can be used as a proxy to probe neural processing speed, as previously argued25. In the next sections, we report the findings from an EEG study that more directly taps into the neural processes that we believe are producing the behavioural pattern.

3a8082e126