Skip to content

calculate_command

dandy.cli.commands.calculate_command

CalculateCommand dataclass

Bases: BaseCommand

help property

add_arguments

Source code in dandy/cli/commands/calculate_command.py 
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
def add_arguments(self, parser):
    parser.add_argument(
        '--model-parameter-count-billions',
        type=int,
        required=True,
        help='The number of billions of model parameters to calculate for.'
    )

    parser.add_argument(
        '--quantization-size-bits',
        type=int,
    )

    parser.add_argument(
        '--tokens',
        type=int,
    )

process

Source code in dandy/cli/commands/calculate_command.py 
28
29
def process(self, *args, **kwargs):
    return self.calculate(**kwargs)

calculate

Source code in dandy/cli/commands/calculate_command.py 
31
32
33
34
35
36
37
38
39
40
41
def calculate(
        self,
        model_parameter_count_billions: int,
        quantization_size_bits: int,
        token_count: int,
) -> float:
    return model_size_and_token_count_for_inference_to_vram_gb(
        model_parameter_count_billions,
        quantization_size_bits,
        token_count
    )