Resource Management

Hmm, if you are going to keep using all of that,

I may need to reconsider …

Roger Riggs
Consulting Member of Technical Staff
Java Products Group
August 10, 2015

Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 2

Program Agenda

1 Goals and Use Cases

2 Resource Model and API
3 Measurement and Monitoring Examples
4 Instrumentation
5 Putting it to Use

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 3

Oracle Confidential –
Resource Monitoring and Management
Goals & Constraints
• Mechanism, not policy
– Just keep count
– Minimal enforcement mechanisms
• Performance
– No impact if not in use
– Minimal impact to track usage (< 5%)
• Resource types
– Files, Sockets, Datagrams, File Descriptors, Threads, CPU, Heap
– Open and close
– Bytes read and written

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 4

Use Case
• Server with dynamic workloads
• Need to monitor resource usage for services or tasks
• Tracking to develop a model of resource use, norms, etc.
• Used for load balancing and throttling of resource use
• Monitoring for excessive use or operating out of bounds

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 5

Model
• ResourceContext
– Set of threads
– For each resource a Meter
• Threads are bound/unbound to a ResourceContext
• Threads consume resources
• Resource use is requested and released, with a resource id
– Request can be approved, denied, or throttled
• Meters accumulate net usage and total usage
• Current thread is mapped to context;
resource type is mapped to meter within the context
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 6
ResourceContext Interface
ResourceContext bindThreadContext();
static ResourceContext unbindThreadContext();
Stream<Thread> boundThreads();

void addResourceMeter(ResourceMeter meter );

boolean removeResourceMeter(ResourceMeter meter);
ResourceMeter getMeter(ResourceType type);
Stream<ResourceMeter> meters();

ResourceRequest getResourceRequest(ResourceType type);

void requestAccurateUpdate(ResourceAccuracy accuracy);

String getName();
void close();

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 7

ResourceContextFactory Interface
ResourceContext create(String name);
ResourceContext lookup(String name);
Stream<ResourceContext> contexts();

ResourceContext getThreadContext();
ResourceContext getThreadContext(Thread thread);
ResourceRequest getResourceRequest(ResourceType type);

ResourceContext getUnassignedContext();
ResourceContext getTotalsContext();

static boolean isEnabled();

Set<ResourceType> supportedResourceTypes();
static ResourceContextFactory getInstance();

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 8

Measure Resource Example
void measure(Runnable run) {
ResourceContextFactory rfactory = ResourceContextFactory.getInstance();
try (ResourceContext rc1 = rfactory.create("rc1")) {
SimpleMeter fopenMeter = SimpleMeter.create(ResourceType.FILE_OPEN);
rc1.addResourceMeter(fopenMeter);
SimpleMeter freadMeter = SimpleMeter.create(ResourceType.FILE_READ);
rc1.addResourceMeter(freadMeter);
rc1.bindThreadContext();
// Consume some resources
run.run();
// Read the meters
System.out.printf("total fopens: %d, bytes read: %d%n",
fopenMeter.getAllocated(), freadMeter.getValue());
} // AutoCloseable
}

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 9

Meters
• SimpleMeter - simply counts
• NotifyingMeter - count with callback
• BoundedMeter - limited to a preset limit
• ThrottledMeter - rate limited to a preset rate
• ResourceType – file [open, read, write],
socket [open, read, write], file descriptor,
threads, cpu, heap, etc.
• ResourceApprover – Interface for callbacks
• ResourceId – extra info about the resource, name and accuracy

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 10

SimpleMeter implements ResourceMeter, ResourceRequest
{
final long getValue();
final long getAllocated();
final ResourceType getType();

// Request handles calling validate and updating the value

final long request(long amount, ResourceId id);

// Validate is overridden to check and modify the value

protected long validate(long previous, long amount, ResourceId id)
throws ResourceRequestDeniedException;

static SimpleMeter create(ResourceType type);

protected SimpleMeter(ResourceType type, ResourceRequest parent);
…
}

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 11

SimpleMeter.request implementation
final long request(long amount, ResourceId id) {
long approved = 0L;
if (amount > 0) {
try { // Allocate
long previous = value.getAndAdd(amount);
approved = validate(previous, amount, id );
} finally {
// If any of the amount was not approved, restore it
long delta = amount - approved;
if (delta != 0) {
value.getAndAdd(-delta);
}
}
} else { // Release
long previous = getValue();
approved = validate(previous, amount, id);
value.getAndAdd(approved);
}

if (approved > 0) {
// Accumulate only the amount allocated
allocated.getAndAdd(approved);
}
return approved;
}
private final AtomicLong value;
private final AtomicLong allocated;

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 12

Monitor Resource Example
static void monitor(Runnable run) {
ResourceContextFactory rfactory = ResourceContextFactory.getInstance();
try (ResourceContext rc1 = rfactory.create("rc1")) {
NotifyingMeter fopenMeter =
NotifyingMeter.create(ResourceType.FILE_OPEN, Example::monitorApprover);
rc1.addResourceMeter(fopenMeter);

NotifyingMeter freadMeter =
NotifyingMeter.create(ResourceType.FILE_READ, Example::monitorApprover);
rc1.addResourceMeter(freadMeter);

rc1.bindThreadContext();

// Consume resources
run.run();
} // AutoCloseable
}

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 13

ResourceApprover Callback
static long monitorApprover(ResourceMeter meter,
long previous,
long amount,
ResourceId resourceId) {
// Caution what you do here should be very limited!
// Because it is called inside of the insides
// Log, accumulate or record
if (amount > 0) {
// consuming
} else {
// releasing
}

return amount; // Return the amount approved

}

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 14

Instrumentation
• Synchronous - library based resources
– Files, Sockets, Datagrams, File Descriptors, Thread creation, etc.
• Asynchronous - polled or event driven
– CPU and Heap

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 15

Instrumentation of Synchronous resources
• Library resources
• Template driven (same as JFR)
• Templates follow the structure of the implemented class
– java.io.File, java.net.Socket, java.net.Datagram, Thread, NIO, etc.
• Templates are applied when first used

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 16

Instrumentation Templates
• Java classes with methods that are wrapped around target methods of a
target class
• Annotations are used to identify target class and methods
• Annotations are used to map types where needed
– i.e. a package private class
• Templates are compiled as normal classes
• The original method contents are 'inlined' into the template

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 17

Instrumentation Templates - II
• ASM used to modify the classfile bytes
• JVMTI is used to redefine the target class with the template applied
• Limitations
– Not a general purpose template mechanism, only supports closed features
– Does not handle exceeding class file limits
– Does not instrument constructors
– Does not support adding fields
– Instrumentation is brittle due to intimate dependency with the instrumented method

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 18

Writing RM instrumentation templates
• References to local variables and class members, methods, and fields are
by name
• Before the resource is consumed (open, read, write)
– code is added to locate the appropriate context and Meter
– to request the amount of resource from the meter (positive amount)
• Template code must anticipate and handle exceptions
• After the resource is consumed
– return any unused portion (negative amount)
– for short reads or thrown exceptions
• Opens and closes must be matched to keep counts meaningful
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 19
Instrumentation Template Example
@InstrumentationTarget("java.io.FileOutputStream")
public final class FileOutputStreamRMHooks {

@InstrumentationMethod
public void write(byte b[]) throws IOException {
if (b == null) {
write(b);
return; // never gets here ...
}
ResourceContext rc = SimpleResourceContext.getThreadContext(Thread.currentThread());
ResourceRequest ra = rc.getResourceRequest(ResourceType.FILE_WRITE);
ResourceId id = ResourceIdImpl.of(path);

int len = b.length;

long approved = 0;
try {
approved = ra.request(len, id);
if (approved < len) {
throw new IOException("Resource limited: insufficient bytes approved");
}
} catch (ResourceRequestDeniedException re) {
throw new IOException("Resource limited: insufficient bytes approved ", re);
}

int numBytesWritten = 0;
try {
write(b);
numBytesWritten = len; // if no exception assume all bytes written
} finally {
// Return any unwritten bytes
ra.request(-(approved - numBytesWritten), id);
}
}
}

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 20

Instrumentation Corner Cases
• APIs may be hard to instrument
– Lack of symmetric methods to hook for open/close – i.e. NIO Async
– The resource identity is not knowable before calling request() – i.e. Socket.accept
• Thread Case
– New thread inherits the ResourceContext of the creator
– Thread is not running when the thread is created
– VM creates the OS level thread and state
• Socket.accept
– The resource id (local socket/port) isn’t known before it completes
– Socket is optimistically accepted and closed if not approved

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 21

Instrumentation of Asynchronous Events
• Retained Heap
– Based on GC events – details to follow
• CPU and heap allocations are sampled periodically
– For each Thread bound to each Resource Context
– CPU time and heap allocation is sampled and accumulated
then reported to respective meter

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 22

Instrumentation of Garbage Collection G1
• G1 collector attributes a good match
– Region based - region contains objects from a single resource context
– Object allocation is thread based, so per allocation overhead is low
• VM Thread structures
– Includes a resource context # (1-255) to associate resource context with thread
– Regions have a field for RC#
– New Thread inherits RC# from creator
• Creating resource context allocates a RC#, the VM’s index for a RC
• Binding a thread to a RC requires a new TLAB (all allocations for same RC)
• Destroying a context reassigns any remaining regions to another RC#
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 23
Events From Garbage Collector
• Completion of GC phases update the retained heap stats for each RC
– Used size in each region is associated with the RC
• Java thread is notified and updated information is read
– Context stats for each RC#, retained heap, accuracy value
– Accuracy value depends on GC phase (LOW, MEDIUM, HIGH, HIGHEST)
• Propagates to the corresponding Resource Context and Meter

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 24

Measurement considerations
• Library resource use has a pretty clear synchronous model
• Heap allocation and retained heap statistics are dynamic
– Normal retained heap accuracy varies, best just after a gc event and declines until the
next one
– Highest Accuracy of retained heap requires a full GC.
– But causes a delay and is just a transient condition
– Can't tell much from a single sample; trends are more useful
• Callbacks for meter updates are concurrent
• Measurement and policy must deal with constant flow updates

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 25

Security and Robustness Risks
– Resource Management can interfere with correct operation behavior and timing
– Callbacks can occur in the middle of *every* I/O operation
– With same protection domain as I/O code; mostly the same as the caller
– Recursion is likely if the callback does any I/O
– Overhead in the callback is a performance hit
– Resource Management API is protected by permission by the SecurityManager

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 26

Resource Policy Considerations
• Synchronous
– Policy can deny usage, exceptions are thrown at call site
– Minimal risk since application expects I/O exception
• Asynchronous - cpu, allocations and retained heap
• No mechanism to deny or throttle
• Risk is high that withholding cpu or memory can lead to further
degradation due to possibly holding locks
• No good way to tell if an application is arbitrarily at a ‘safe’ point to stop
• Best mechanism is the existing work or task scheduler

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 27

Summary
• Low performance impact on I/O (under 2%) - only when in use
• Template driven instrumentation
• Retained heap instrumentation integrated with G1 collector
• Dynamic nature of resource data adds complexity to policy formulation

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 28

References and Credits
• Javadoc
http://docs.oracle.com/javase/8/docs/jre/api/management/rm/index.html

• Resource Management is Commercial Feature of 8u40

http://www.oracle.com/technetwork/java/javase/documentation/otn-bcl-02april2013-1966219.pdf

• Credits
– Stefan Johansson – Hotspot GC
– John Coomes – Hotspot GC
– Staffan Larson – JVMTI and Instrumentation Template
– Brian Burkhalter – I/O instrumentation
– Charlie Hunt – Performance
– WLS Team for motivation and requirements
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 29