Use argparse to parse a list of objects
I have a program with a function that takes a class initializer and list of objects. Each object consists of 3 variables id, value, and tag.
class Package(): def __init__(self, id, value, name): if (value I'm trying to implement argparse in main() so I can run the program by doing something like: python foo.py "int0 [(int1, float1, int2), (int3, float2, int4) . ]" . The number of objects in the list is variable and depends on the user input.
initializer = num0 //First package object package.id = num1 package.value = num2 package.tag = num3 //Second package object package.id = num4 package.value = num5 package.tag = num6 3 Answers 3
You can make a custom argument type and use ast.literal_eval() to parse the value.
import argparse from ast import literal_eval class Package(): def __init__(self, id, value, tag): if (value Now if you would run the script, you would get a list of Package class instances printed:
$ python test.py --packages="[(1, 1.02, 3), (40, 2.32, 11)]" [[, ]] Cool! I've never used ast.literal_eval() . When return [Package(*item) for item in data] is called, will all the arguments be stored in the right package variable (package.id, package.value, package.tag)?
@Brosef yup, these are Package instances in the list with all of the 3 fields initialized. Hope this is close to what you were looking for.
I would prefer to be a bit more explicit and use a custom action:
import argparse class PackageAction(argparse.Action): def __init__(self, *args, **kwargs): super(PackageAction, self).__init__(*args, **kwargs) self.nargs = 3 def __call__(self, parser, namespace, values, option_string): lst = getattr(namespace, self.dest, []) or [] a, b, c = values lst.append(Package(int(a), float(b), int(c))) setattr(namespace, self.dest, lst) class Package(object): def __init__(self, foo, bar, baz): self.foo = foo self.bar = bar self.baz = baz def __repr__(self): return 'Package(%r, %r, %r)' % (self.foo, self.bar, self.baz) parser = argparse.ArgumentParser() parser.add_argument('--package', action=PackageAction) print(parser.parse_args()) The usage here would look something like:
$ python packager.py --package 1 2 3 --package 4 5 6 Namespace(package=[Package(1, 2.0, 3), Package(4, 5.0, 6)]) One benefit is that you get slightly better default error handling . e.g.:
$ python ~/sandbox/test.py --package 1 2 3 --package 4 5 usage: test.py [-h] [--package PACKAGE PACKAGE PACKAGE] test.py: error: argument --package: expected 3 argument(s) Of course, you can modify to suit your purposes -- specifically it would probably be good to provide a little extra error handling to __call__ . e.g. you could do something like
parser.error('--package requires an int float and int') if the user passed bad strings. You could also provide better variable names 🙂
Here's my nomination; it uses a plain parser, and puts the customization in the Package class.
It would be called with something like:
python prog.py -p 0 1 2 --package 2 3 4 where -p or --package is followed by 3 values, and may be repeated ( action is 'append'). nargs=3 ensures that each -p is followed by 3 values (otherwise the parser raises an error). Converting those values to numbers (and raising errors) is the responsibility of the Package class. The class already has a check on non-negative value .
import argparse class Package(): def __init__(self, id, value, tag): # 3 inputs - numbers, but equivalent strings are accepted # may add more value validation self.id = int(id) self.value = float(value) if self.value A sample run of the test cases is:
2030:~/mypy$ python stack34823075.py Namespace(package=[]) [] Namespace(package=[['1', '2', '3']]) [Package (1, 2.0, 3)] Namespace(package=[['0', '1', '2'], ['2', '3', '4']]) [Package (0, 1.0, 2), Package (2, 3.0, 4)] usage: stack34823075.py [-h] [-p ID Value tag] optional arguments: -h, --help show this help message and exit -p ID Value tag, --package ID Value tag package parameters; may repeat Note how the metavar affects the help display. The Package __repr__ method produces a nice list display.
Example of a run with non-numeric tag :
2038:~/mypy$ python stack34823075.py -p 1 2.3 tag Namespace(package=[['1', '2.3', 'tag']]) Traceback (most recent call last): File "stack34823075.py", line 31, in print main(sys.argv[1:]) File "stack34823075.py", line 20, in main packages = [Package(*v) for v in args.package if v is not None] File "stack34823075.py", line 10, in __init__ self.tag = int(tag) ValueError: invalid literal for int() with base 10: 'tag' A special type function does not work well here. It would be applied to each of the 3 strings, individually, not as a group.
A custom Action class could handle the 3 values, converting each to int , float , int . But even there I'd prefer to pass them to Package , e.g.
def __call__(self, namespace, dest, values): # store_action style new_value = Package(*values) setattr(namespace, dest, new_value) # store action But since packages = [Package(*v) for v in args.package] is so simple, I don't see much point in customizing the parser or its actions.
Passing a List to Python From Command Line
I would like to make my python script run from the command line when supplies with some arguments. However, one of the arguments should be a list of options specific to one segment of the script. Would string parsing be the only way to do this by actually constructing the list after the "command line list" string is split from commas? If so, how would you go about that? Example: -details=['name', 'title', 'address']
Is your question how to go about "constructing the list after the "command line list" string is split from commas?" That's how I've interpreted your question, please clarify
did you ever find a solution built in to the argparse command library or something like that, that doesn't require potentially hacky custom code?
4 Answers 4
import sys, ast, getopt, types def main(argv): arg_dict=<> switches= singles=''.join([x[0]+':' for x in switches]) long_form=[x+'=' for x in switches] d= try: opts, args = getopt.getopt(argv, singles, long_form) except getopt.GetoptError: print "bad arg" sys.exit(2) for opt, arg in opts: if opt[1]+':' in d: o=d[opt[1]+':'][2:] elif opt in d.values(): o=opt[2:] else: o ='' print opt, arg,o if o and arg: arg_dict[o]=ast.literal_eval(arg) if not o or not isinstance(arg_dict[o], switches[o]): print opt, arg, " Error: bad arg" sys.exit(2) for e in arg_dict: print e, arg_dict[e], type(arg_dict[e]) if __name__ == '__main__': main(sys.argv[1:]) Command line:
python py.py --l='[1,2,3,[1,2,3]]' -d "" --tu='(1,2,3)' args: ['--l=[1,2,3,[1,2,3]]', '-d', "", '--tu=(1,2,3)'] tu (1, 2, 3) di li [1, 2, 3, [1, 2, 3]]
This code snippet will take short or long command switches like -l or --li= and parse the text after the switch into a Python data structure like a list, tuple or a dict. The parsed data structure ends up in a dictionary with the long-form switch key.
Using ast.literal_eval is relatively safe. It can only parse python data definitions.
argparse is nice for this, it's in the standard library as of 2.7 and 3.2 but otherwise a pip install away.
Your main concern of specifying a variable-length list can be addressed by making the list interpreted as a single argument in the shell by using quotes (could depend on your shell I suppose):
% python prog.py 'name title address' spam import sys my_list = sys.argv[1].split() # my_list is ['name', 'title', 'address'] if 'name' in my_list: do_something() or similar. Use an argument with split to delimit your list:
% python prog.py "you're a foo, lift the bar" my_list = [x.strip() for x in sys.argv[1].split(',')] # my_list is ["you're a foo", "lift the bar"] But please use argparse instead; especially if you want to use use -c style flags.
One way to interpret your question is:
"I'm already using argparse, since that's the sensible way to interpret command line arguments in Python. How do I specify that some options are within a specific category?"
In your question you've shown an example of something the shells I use of would choke on;
% python prog.py -v -details=['name', 'title', 'address'] --quickly -t 4 wouldn't make it to python to be parsed because they'd use spaces to separate arguments and might use [ and ] as shell syntax.
I suggest the following instead
% python prog.py -v --details name title address --quickly -t 4 import argparse parser = argparse.ArgumentParser() parser.add_argument('-v', action='store_true') parser.add_argument('--details', nargs='*') parser.add_argument('--quickly', action='store_true') parser.add_argument('-t') args = parser.parse_args() #args is Namespace(details=['asdf', 'a', 'a'], quickly=False, t='4', v=True) details = args.details #details is ['asdf', 'a', 'a'] Now, as per your question, you didn't have to do the string parsing yourself.
Yes, the asker has already figured out the .split() bit, based on the question. It is the next step that is causing problems.
@CharlieParker looks like I wanted to get rid of the leading space in " lift the bar" that would otherwise have been there
Yes, argparse is your best bet, and if you want to provide a list of values to one of your named arguments, it looks like this (the nargs parameter is the key to this):
>>> import argparse >>> arg_parser = argparse.ArgumentParser() >>> arg_parser.add_argument('--details', nargs='*', type=str, default=[], help='a list of the details') # your args on the command line like this example >>> the_args = arg_parser.parse_args("--details 'name' 'title' 'address'".split()) >>> print the_args.details ["'name'", "'title'", "'address'"]) I really like the-wolf's approach using variable length collections as explicit string arguments.
In my opinion, nargs='*' has noteworthy drawbacks: when attempting to collect strings as positional arguments (at least one string must be present), or if you are trying to use subparsers, you will find that nargs='*' and nargs='+' use greedy completion, and they don't seem to stop consuming for any good reasons. Even if syntax for an optional argument or a number comes up, the string() type will keep consuming. (This gets harder to anticipate with subparsers).
Best case, arguments placed after (positional and optional) are being ignored, and worse, you are likely passing corrupt data types to the argparse array.
We should be able to define a custom ActionType that is looking for a quoted string. If it finds one, then we adapt the-wolf's examples (nearly verbatim it seems).
This keep things clean in argparse and makes general use of variable collections much less bitchy.