July 23 webex
8 views
Skip to first unread message
Michael Wong
Jul 23, 2018, 3:08:08 PM7/23/18
to SG5 - Transactional Memory
Apology for the late reminder. Hans, Herb and I are on the call. Just want to see if anyone else can join. Thanks.
Start Time: Monday, July 23 , 2018, 12:00 PM US Pacific Time (07:00 PM in GMT)
End Time: 1:00 PM US Pacific Time (duration: one hour)
your phone muted whenever possible. If your phone does not have a mute
button, the bridge will mute or un-mute your line if you dial *6.
The current secretary rota list is (the person who took notes at the last meeting is moved to the end)
Maged, Jens, Victor,, Hans, Michael Scott, Michael Spear , Michael W
Agenda:
1. Opening and introductions
1.1 Roll call of participants
1.2 Adopt agenda
1.3 Approve minutes from previous meeting, and approve publishing previously approved minutes to ISOCPP.org
1.4 Review action items from previous meeting (5 min)
1.5 Call schedules (please add your away days)
2. Main issues (50 min)
2.1 Continue discussion on future of TM logistics
Continue on minimal TM lite proposal based on Herb Sutter's ideas (Herb to come on)
Tim Sweeney has been occassionally pinging us or talking about us and has replied to our requests:
From Jan 29 meeting:
In my view, this is going the wrong way. Here's a summary of reasons, and an alternative path forward.
- Transactional memory (how do we track all reads and writes to all shared memory?)
- Persistence (how do we allocate, find, persist, and manage data long-term without corruption?)
- Garbage collection (how do we find out what memory is actively being used?)
-
ABI (how do we provide interface and data backwards-compatibility over
multiple program invocations and even across platforms?)
- Reflection (what is the format of all of our data?)There are several different ways that SG5 could approach this topic.
The
current approach is to try to expose transactional memory at the
language level. This is difficult, expensive, not fully orthogonal to
the other topics above. More generally, it seems aloof to the C++ way,
which is to expose general abstractions to programmers so we can
implement specific features. Examples of abstractions include functions
(1960), templates (1990), and reflection (2020?)
My
view is: Give us a great reflection spec, and we'll do the other things
ourselves in libraries. Doing this in libraries would be a good thing
because:
- Developers can experiment and discover what works best, as opposed to mandating a solution that's only roughly prototyped.
-
Designing containers for transactional, persistent, garbage-collected,
binary-forward-compatible containers leads to very different designs
than std.
- New transactional, persistent, garbage-collected
code will need to coexist and interoperate with existing libraries, so
fine-grained control will be needed -- which is natural with a library
solution to these problems, and unnatural with cross-cutting language
features.
- Reflection provides the full toolset needed to
build the features above. Transactions via new templated container
types; persistence via anything from serialization to patching memory to
upgrade versions in-place; garbage collection via metadata; and ABI
compatibility by automatically creating forward-compatible wrappers and
adapters.
A minimalist alternative for SG5 is
to simply bless (via std extensions) the kind of
accelerated-but-not-guaranteed restricted transactional memory of
Intel's TSX and similar related proposals. These are well-understood
low-level features that libraries can build on to implement full
transactional memory and the other things, on an opt-in basis.
From Tim Sweeney to SG7 reflection on 12/28/17
With just reflection, and no reliance on a future generative C++
proposal, we can generate specialized functions that mimic the behavior
of constructors and destructors but are customized for special usage
cases. For example: a "deserializing constructor" to generate a new
instance of a class from a stream, or helper constructors for optimized
garbage-collection schemes.
Thanks
for P0670! Its coverage of static reflection on functions and lambdas
in particular looks like exactly what's needed for exposing lambda
captures to a garbage collector.
As an aside, I agree
with the importance of exposing function parameter names. These will
all be important in doc tools, RPC frameworks, and marshalling from
scripting languages built on C++ reflection
One thing we can't with
reflection alone is member-specific customization smart pointers to
classes. For example, given "gc_smart_pointer<t> p", we can
implement "gc_smart_pointer<t>::operator->()", but it has to behave uniformly for all types.
Could
we have a per-class overloadable templated variant of
operator->(t&) which receives a meta object describing the
particular member being accessed? Then it can customize its behavior
according to the type and member being accessed. This would be useful
for optimized garbage-collection schemes (where accessing a POD can be
optimized compared to a garbage-collector-managed type); software
transactional memory schemes (which would like to store data in a class
as a simple type, but access it using a wrapper type), marshaling layers
that connect C++ to scripting languages; etc.
From Tim Sweeney to SG7
Will P0194 be extended to support lambdas, and specifically reflecting on the number and type of lambda captures?
Reflecting
on lambda captures is critically important in the case of implementing a
garbage collector on top of standard C++, without hardcoding knowledge
of memory layout or other things.
Background
In
standard C++, a general-purpose garbage collector can be implemented on
top of smart pointers with reference counting. Any allocation with a
nonzero reference count is treated as a GC root. To get from this
starting point to real garbage collection, we can provide a mechanism
for certain types (such as containers) which are themselves
heap-allocated and reference counted, to release the reference counts of
their contents once they're initialized.
This
can be automated by replacing "new t(parms)" with
"newref<t>(parms)", which allocations memory, calls a constructor,
and ensures smart pointers release their reference counts immediately
rather than in their destructor. This approach breaks the
reference-counting cycles for those heap-allocated types, while ensuring
everything on the stack remains a GC root.
Using
this approach, I have a neat concurrent, nonblocking garbage collector
up and running on top of standard C++17. Without static reflection,
this requires manually implementing reference-count-releasing functions
for essential types.
With C++2a, reflection
could make it completely automatic. So, instead of using raw pointers
and new, you use a smart pointer and newref, and get free, safe GC
within standard C++. For this to work well, we'd need to reflect lambda
captures. If we can't reflect lambda captures, then we are nearly
certain that they'll be held forever due to reference-counting cycles,
because a lambda's purpose is often to manipulate an object it's stored
in. Thus the lambda pins the object, and the object contains the
lambda, so it's never released. Eager functional languages like ML
require garbage collection solely because of these cyclic references
between containers and lambdas within them.
Aside: Garbage Collection in Future C++
I
believe C++ will fundamentally require concurrent GC in order to scale
to many-threaded programs with complex data dependencies and
asynchronous execution. Objects, lambdas, and futures interact in so
many subtle ways that manual memory management seems intractable.
(Unreal Engine has relied on a hand-coded C++ garbage collector since
1998.)
I feel the N2670 garbage collection
track is neither tenable nor desirable. C++ isn't about heavyweight
runtime plumbing; it's about giving the programmer control, and relying
on standard and user libraries to solve common problems. Give us a
thorough version of P0194 and we'll have garbage collection soon
enough. And it won't be a conservative kluge that stops all threads and
scans all stacks and memory looking for pointer-like things; it will be
a standard C++ implementation which users can opt-in to, while
remaining safe and composable with all other libraries, whether they use
GC or not.
| 12/19/17 |
| |||
Hi Tim,
On 18.12.17 23:08, tim.s...@epicgames.com wrote:Will P0194 be extended to support lambdas, and specifically reflecting on the number and type of lambda captures?
Reflecting on lambda captures is critically important in the case of implementing a garbage collector on top of standard C++, without hardcoding knowledge of memory layout or other things.
Wow, super-interesting. I'd love to see a CppCon talk on that, to
better understand that idea.
And while P0194 excludes lambdas, its follow-up paper P0670 includes them and will appear in LEWG + EWG for the next C++ standards meeting, i.e. it's on track to catch up with P0194.
Could you check whether it does what you need?
Cheers, Axel.
| 12/20/17 |
| |||
From Herb Sutter:
I just got this in email from Tim Sweeney:
Transactional memory: Supporting this natively is totally crazy in C++, which has far too much low-level mutable state by default and will be hopelessly inefficient, and uses mutable containers whose internal implementations (reading and writing lots of state) will create vast false conflicts. Rather, we should approach this two ways: ISO C++ should quickly adopt and expose failable Intel TSX-style transactions for small, low-level operations; …
This matches my encouragement to the group to please consider “small local transactions” that are just some small fixed number of memory-only operations… that avoids the entire composability/annotation problem and immediately enables a whole class of lock-free data structures that need (only) multi-word/non-contiguous CAS.
2.2: Interaction with Executors and Synchonized proposal
https://groups.google.com/a/isocpp.org/forum/#!topic/tm/jG9XPJetNkc
The last discussion has us considering an alternative lambda form.
See Paper emailed out on Lambda proposal
https://docs.google.com/document/d/1ICmcrCdigq3ataoM2Jl7m19h_Sa3aE3KfU6AVkPyT-4/edit#
2.3 future issues list:
1. llvm synchronized blocks
2. more smart ptrs?how fast can atomics and smart ptrs be outside tx if they have to interact with tx (for world that does not care about tx), the atomic nature of smart ptrs as a way towards atomics inside atomic blocks
3. more papers?
4. Issue 1-4 paper updates to current TM spec
5. std library
2.4 Discuss defects if any work done since last call
Issue 1: https://groups.google.com/a/isocpp.org/forum/#!topic/tm/SMVEiVLbdig
Issue 2: https://groups.google.com/a/isocpp.org/forum/#!topic/tm/Th7IFxFuIYo
Issue 3:https://groups.google.com/a/isocpp.org/forum/#!topic/tm/CXBycK3kgo0
Issue 4: https://groups.google.com/a/isocpp.org/forum/#!topic/tm/Ood8sP1jbCQ
3. Any other business
4. Review
4.1 Review and approve resolutions and issues [e.g., changes to SG's working draft]
N4513 is the official working draft (these links may not be active yet until ISO posts these documents)
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4513.pdf
N4514 is the published PDTS:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4514.pdf
N4515 is the Editor's report:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4514.html
Github is where the latest repository is (I have updated for latest PDTS published draft from post-Leneaxa):
https://github.com/cplusplus/transactional-memory-ts
Bugzilla for filing bugs against TS:
https://issues.isocpp.org/describecomponents.cgi
4.2 Future backlog discussions:
4.2.1 Write up guidance for TM compatibility for when TM is included in C++ standard (SG5)
4.2.2 Continue Retry discussion
https://groups.google.com/a/isocpp.org/forum/?hl=en&fromgroups#!topic/tm/qB1Ib__PFfc
https://groups.google.com/a/isocpp.org/forum/#!topic/tm/7JsuXIH4Z_A
4.2.3 Issue 3 follow-up
Jens to follow up to see if anything needs to be done for Issue 3.
4.2.5 Future C++ Std meetings:
2018 06-04 RAP C++ Std meeting
4.3 Review action items (5 min)
5. Closing process
5.1 Establish next agenda
5.2 Future meeting
Next call: TBD
Michael Wong
Jul 23, 2018, 3:28:05 PM7/23/18
to SG5 - Transactional Memory
On Monday, July 23, 2018 at 3:08:08 PM UTC-4, Michael Wong wrote:
Apology for the late reminder. Hans, Herb and I are on the call. Just want to see if anyone else can join. Thanks.Start Time: Monday, July 23 , 2018, 12:00 PM US Pacific Time (07:00 PM in GMT)End Time: 1:00 PM US Pacific Time (duration: one hour)
With large numbers of participants, audio interference can be a problem. Please try to keep
your phone muted whenever possible. If your phone does not have a mute
button, the bridge will mute or un-mute your line if you dial *6.
The current secretary rota list is (the person who took notes at the last meeting is moved to the end)
Maged, Jens, Victor,, Hans, Michael Scott, Michael Spear , Michael W
Agenda:
1. Opening and introductions
1.1 Roll call of participants
Hans, Herb, Victor, Michael W
1.2 Adopt agenda
1.3 Approve minutes from previous meeting, and approve publishing previously approved minutes to ISOCPP.org
1.4 Review action items from previous meeting (5 min)
1.5 Call schedules (please add your away days)
2. Main issues (50 min)2.1 Continue discussion on future of TM logistics
Herb illustrating problem concerns:
Hans saids it is solving slightly different problem
1. generic programming
2. ordinary reads and writes
Herb like #2 to help with solving #1
Hans: synhronized blocks can fall back on other things
Hans: where is the HW world
Michael: Paul M wants me to contact this guy
ARM: Stephen Diestelhorst connect with him
power9 will continue, xeon phi has removed it,
Herb: software will drive HW to support it
Herb, intuition is to make it unbounded, get guaranteed lock free
are you going to be happy with small restrictive set:
Herb: we can grow this set
give a strong restriction and later relax the rules
Michael: need this to integrate well with rest of the C++ language ( a strong Bjarne requirement) , unlike the TS which seems to standalone
Hans: yes this wll fit well
Hans: except the ...
Herb: constexpr is viral
Scott, Michael
Jul 23, 2018, 3:28:20 PM7/23/18
to t...@isocpp.org
Sorry - I’m between planes and can’t join in.
- Michael
--
You received this message because you are subscribed to the Google Groups "SG5 - Transactional Memory" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tm+unsu...@isocpp.org.
To post to this group, send email to t...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/tm/.
Hans Boehm
Jul 23, 2018, 5:04:39 PM7/23/18
to t...@isocpp.org
Part B of minutes:
[ completing my statement from part A: ...except the prohibition on function calls. ]
Herb: Generate action items
MW: either reduce existing MS proposal, or start brand new.
Herb: Generate cooperative proposal.
Viral annotation is show-stopper. Would like a pearl that we can grow.
Victor: Want to find something that can fit cleanly into C++. Starting with a small useful piece would be
good. Grow it later.
Herb: Let's update the proposal. Get something within 18 months. Could be parallel to current TS.
Feature cut-off for 20 is in next meeting or two. Not aiming for C++20.
Victor: Mostly strip down existing proposal.
Herb: Want atomic { } clean syntax.
Start with statement level.
Victor,Herb: Not guaranteed lock-free.
Herb: Against hard limit, unless there is a killer example. Trait only with strong motivation.
Victor: To talk to Mike Spear about this.
Tentative consensus that this stripped-down construct should be truly atomic. Currently this won't matter since synchronization inside transactions will be disallowed. Can re-add synchronized blocks.
Michael Wong: can't make August 6 meeting.
Keep August 6 slot, just chat informally.
--
Michael Spear
Aug 6, 2018, 2:10:12 PM8/6/18
to t...@isocpp.org
Hi Everyone,
Do we have a call today?
- Mike
Victor Luchangco
Aug 6, 2018, 2:57:01 PM8/6/18
to t...@isocpp.org
Yes, I think so. Michael Wong can't make it, but I think he said that his WebEx will work anyway. If that doesn't work, then I guess we can just postpone, but I think it would be good to touch base again about what the plan is, especially since you weren't able to make the call a couple of weeks ago. If we run out of things to talk about, we can just call the meeting early.
- Victor
Reply all
Reply to author
Forward
0 new messages