6.1.5.6. hat.profiler

Profilers widely used for perf in HAT.

6.1.5.6.1. Profiler

BaseProfiler

If you wish to write a custom profiler, you should inherit from this class.

PassThroughProfiler

This class should be used when you don't want the (small) overhead of profiling.

SimpleProfiler

This profiler simply records the duration of actions (in seconds) and reports the mean duration of each action and the total time spent over the entire training run.

PythonProfiler

This profiler uses Python's cProfiler to record more detailed information about time spent in each function call recorded during a given action.

GPUMemoryProfiler

PyTorchProfiler

This profiler uses PyTorch's Autograd Profiler and lets you inspect the cost of.

CPUMemoryProfiler

StageCPUMemoryProfiler

6.1.5.6.2. API Reference

class hat.profiler.BaseProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None)

If you wish to write a custom profiler, you should inherit from this class.

describe() → None

Log a profile report after the conclusion of run.

profile(action_name: str) → None

Yield a context manager to encapsulate the scope of a profiled action.

Example:

with self.profile('load training data'):
    # load training data code

The profiler will start once you’ve entered the context and will automatically stop once you exit the code block.

setup(stage: Optional[str] = None, local_rank: Optional[int] = None) → None

Execute arbitrary pre-profiling set-up steps.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

teardown(stage: Optional[str] = None) → None

Execute arbitrary post-profiling tear-down steps.

Closes the currently open file and stream.

class hat.profiler.CPUMemoryProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None)
start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary()

Create profiler summary in text format.

class hat.profiler.GPUMemoryProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None)
start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary()

Create profiler summary in text format.

class hat.profiler.PassThroughProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None)

This class should be used when you don’t want the (small) overhead of profiling. The Trainer uses this class by default.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

class hat.profiler.PyTorchProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None, group_by_input_shapes: bool = False, emit_nvtx: bool = False, export_to_chrome: bool = True, row_limit: int = 20, sort_by_key: Optional[str] = None, record_functions: Optional[Set[str]] = None, step_function: Optional[str] = None, record_module_names: bool = True, **profiler_kwargs: Any)

This profiler uses PyTorch’s Autograd Profiler and lets you inspect the cost of.

different operators inside your model - both on the CPU and GPU

参数

  • dirpath – Directory path for the filename.

  • filename – If present, filename where the profiler results will be saved instead of printing to stdout. The .txt extension will be used automatically.

  • group_by_input_shapes – Include operator input shapes and group calls by shape.

  • emit_nvtx –

    Context manager that makes every autograd operation emit an NVTX range Run:

    nvprof --profile-from-start off -o trace_name.prof -- <regular command here>

    To visualize, you can either use:

    nvvp trace_name.prof
torch.autograd.profiler.load_nvprof(path)

  • export_to_chrome – Whether to export the sequence of profiled operators for Chrome. It will generate a .json file which can be read by Chrome.

  • row_limit – Limit the number of rows in a table, -1 is a special value that removes the limit completely.

  • sort_by_key – Attribute used to sort entries. By default they are printed in the same order as they were registered. Valid keys include: cpu_time, cuda_time, cpu_time_total, cuda_time_total, cpu_memory_usage, cuda_memory_usage, self_cpu_memory_usage, self_cuda_memory_usage, count.

  • record_functions – Set of profiled functions which will create a context manager on. Any other will be pass through.

  • step_function – Profiled function, which means that after the function ends, the current iteration is end and next iteration will start.

  • record_module_names – Whether to add module names while recording autograd operation.

  • profiler_kwargs – Keyword arguments for the PyTorch profiler. This depends on your PyTorch version

  • Raises –

    Exception:

    If arg sort_by_key is not present in AVAILABLE_SORT_KEYS. If arg schedule is not a Callable. If arg schedule does not return a torch.profiler.ProfilerAction.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

teardown(stage: Optional[str] = None) → None

Execute arbitrary post-profiling tear-down steps.

Closes the currently open file and stream.

class hat.profiler.PythonProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None, line_count_restriction: float = 1.0, output_filename: Optional[str] = None)

This profiler uses Python’s cProfiler to record more detailed information about time spent in each function call recorded during a given action. The output is quite verbose and you should only use this if you want very detailed reports.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

teardown(stage: Optional[str] = None) → None

Execute arbitrary post-profiling tear-down steps.

Closes the currently open file and stream.

class hat.profiler.SimpleProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None, extended: bool = True)

This profiler simply records the duration of actions (in seconds) and reports the mean duration of each action and the total time spent over the entire training run.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

class hat.profiler.StageCPUMemoryProfiler(profile_action_name: str, leaks: bool = True, dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None)
describe() → None

Log a profile report after the conclusion of run.

profile(action_name: str)

Yield a context manager to encapsulate the scope of a profiled action.

Example:

with self.profile('load training data'):
    # load training data code

The profiler will start once you’ve entered the context and will automatically stop once you exit the code block.