os.path.argparse - optional startdir argument

[Wolfgang Maier]

Dear all,

currently, os.path.abspath(somepath) is, essentially, equivalent to

os.path.normpath(os.path.join(os.getcwd(),path)).

However, I'd find it useful, occasionally, to be able to specify a
starting directory other than the current working directory.

One such situation is when reading a config file of an application: if
you encounter a relative link in such a file, you'll typically want to
transform it into an absolute path using that application's working
directory as opposed to your own one.

My suggestion would be to add an optional startdir argument to abspath,
which, when provided would be used instead of os.getcwd(). If startdir
itself is not an absolute path either, it would be turned into one
through recursion.

Currently, you have to write:

os.path.normpath(os.path.join(startdir, path))
or even
os.path.normpath(os.path.join(os.path.abspath(startdir), path))

instead of the proposed:

os.path.abspath(path, startdir)

Before posting I checked the bug tracker and found that this idea has
been brought up years ago (http://bugs.python.org/issue9882), but not
pursued further.
The patch suggested there is a bit of an oversimplification, but I have
my own one, which I could provide if someone's interested.
For issue9882 it was suggested to bring it up on python-ideas, but to
the best of my knowledge that was never done, so I'm doing it now.

Thoughts ?

Wolfgang

[Wolfgang Maier]

Just realized my typo: I meant os.path.abspath in the title - don't know
what I was thinking about when I typed that

[Juancarlo Añez]

On Thu, Jul 24, 2014 at 9:15 AM, Wolfgang Maier <

Post by Wolfgang Maier
os.path.normpath(os.path.join(os.getcwd(),path)).
However, I'd find it useful, occasionally, to be able to specify a
starting directory other than the current working directory.

os.path.normpath(os.path.join(config_dir, path))

Better yet, use the pathlib module.

Cheers,

[Wolfgang Maier]

Post by Juancarlo Añez
On Thu, Jul 24, 2014 at 9:15 AM, Wolfgang Maier
os.path.normpath(os.path.join(__os.getcwd(),path)).
However, I'd find it useful, occasionally, to be able to specify a
starting directory other than the current working directory.
os.path.normpath(os.path.join(config_dir, path))

As I said, I'm aware of this, but it's ugly and even uglier if you have
to turn config_dir into an absolute path itself.

Post by Juancarlo Añez
Better yet, use the pathlib module.

As it stands, the pathlib module is only provisional plus, IMO, kind of
overkill for a simple task like that.

Post by Juancarlo Añez
Cheers,
Juancarlo *Añez*

[Juancarlo Añez]

On Thu, Jul 24, 2014 at 11:00 AM, Wolfgang Maier <

Post by Wolfgang Maier
As it stands, the pathlib module is only provisional plus, IMO, kind of
overkill for a simple task like that.

https://docs.python.org/3/library/pathlib.html

The pathlib module is "*New in version 3.4"*. There's an implementation for
previous versions of Python in PyPi.

https://pypi.python.org/pypi/pathlib

The pathlib module is not overkill, as it provides the same functionality
as os.path, but in a more OO and syntactically simpler form:

(configpath / filepath).resolve()

Cheers,

[Terry Reedy]

Post by Wolfgang Maier

Post by Juancarlo Añez
Better yet, use the pathlib module.

Thank for the reminder. I took a better look at it.

Post by Wolfgang Maier
As it stands, the pathlib module is only provisional plus,

'Provisional' means that there *could* be a few api changes that would
break code. The module is not going away.

Post by Wolfgang Maier
IMO, kind of overkill for a simple task like that.

Overkill?

import pathlib as path
import os.path as path

are equally easy

The 'simple task' combines joining, normalizing, and 'absoluting'.
pathlib.Path joins, Path.resolve normalizes and 'absolutes'. Together
they combine the functions of os.path.join, os.path.abspath and
os.path.normpath, with a nicer syntax, and with OS awareness.

Post by Wolfgang Maier

Post by Juancarlo Añez

path.Path('../../../Python27/lib', 'ast.py').resolve()

WindowsPath('C:/Programs/Python27/Lib/ast.py')

If one starts with a Path object, as would be typical, one can use '/'
to join, as JuanCarlo mentioned.

Post by Wolfgang Maier

Post by Juancarlo Añez

base = path.Path('.')
(base / '../../../Python27/lib' / 'ast.py').resolve()

WindowsPath('C:/Programs/Python27/Lib/ast.py')

[Juancarlo Añez]

On a related topic...

What's missing in Python 3.4, is that most modules with functions or
methods that take file names or file paths as parameters are not
pathlib-aware, so a mandatory str(mypahtlibpath) is required.

For example, you cannot do:

f = open(Path(_file__) / 'app.conf')

It will fail.

But pathlib as part of the standard lib is new, so it's OK.

It will take time how know where in the module dependency hierarchy it
should belong.

Cheers,

Post by Juancarlo Añez
Better yet, use the pathlib module.
Thank for the reminder. I took a better look at it.
As it stands, the pathlib module is only provisional plus,
'Provisional' means that there *could* be a few api changes that would
break code. The module is not going away.
IMO, kind of overkill for a simple task like that.
Overkill?
import pathlib as path
import os.path as path
are equally easy
The 'simple task' combines joining, normalizing, and 'absoluting'.
pathlib.Path joins, Path.resolve normalizes and 'absolutes'. Together they
combine the functions of os.path.join, os.path.abspath and
os.path.normpath, with a nicer syntax, and with OS awareness.

path.Path('../../../Python27/lib', 'ast.py').resolve()

WindowsPath('C:/Programs/Python27/Lib/ast.py')
If one starts with a Path object, as would be typical, one can use '/' to
join, as JuanCarlo mentioned.

base = path.Path('.')
(base / '../../../Python27/lib' / 'ast.py').resolve()

WindowsPath('C:/Programs/Python27/Lib/ast.py')
--
Terry Jan Reedy
_______________________________________________
Python-ideas mailing list
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Ryan]

Instead, though, you'd do:

f = (Path(_file__) / 'app.conf').open()

https://docs.python.org/3/library/pathlib.html#pathlib.Path.open

Post by Juancarlo Añez
On a related topic...
What's missing in Python 3.4, is that most modules with functions or
methods that take file names or file paths as parameters are not
pathlib-aware, so a mandatory str(mypahtlibpath) is required.
f = open(Path(_file__) / 'app.conf')
It will fail.
But pathlib as part of the standard lib is new, so it's OK.
It will take time how know where in the module dependency hierarchy it
should belong.
Cheers,

Post by Juancarlo Añez
Better yet, use the pathlib module.
Thank for the reminder. I took a better look at it.
As it stands, the pathlib module is only provisional plus,
'Provisional' means that there *could* be a few api changes that

would

Post by Juancarlo Añez
break code. The module is not going away.
IMO, kind of overkill for a simple task like that.
Overkill?
import pathlib as path
import os.path as path
are equally easy
The 'simple task' combines joining, normalizing, and 'absoluting'.
pathlib.Path joins, Path.resolve normalizes and 'absolutes'. Together

they

Post by Juancarlo Añez
combine the functions of os.path.join, os.path.abspath and
os.path.normpath, with a nicer syntax, and with OS awareness.

path.Path('../../../Python27/lib', 'ast.py').resolve()

WindowsPath('C:/Programs/Python27/Lib/ast.py')
If one starts with a Path object, as would be typical, one can use

'/' to

Post by Juancarlo Añez
join, as JuanCarlo mentioned.

base = path.Path('.')
(base / '../../../Python27/lib' / 'ast.py').resolve()

WindowsPath('C:/Programs/Python27/Lib/ast.py')
--
Terry Jan Reedy
_______________________________________________
Python-ideas mailing list
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--
Juancarlo *Añez*
------------------------------------------------------------------------
_______________________________________________
Python-ideas mailing list
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Juancarlo Añez]

Post by Ryan
f = (Path(_file__) / 'app.conf').open()

Indeed, that solves the "right place in the module dependency hierarchy"
thing, and it even has an "econding=" kwarg!

I hadn't paid attention to it. Sorry.

Problem solved!

Thanks!

[Nick Coghlan]

Post by Juancarlo Añez
On a related topic...
What's missing in Python 3.4, is that most modules with functions or

methods that take file names or file paths as parameters are not
pathlib-aware, so a mandatory str(mypahtlibpath) is required.

Post by Juancarlo Añez
f = open(Path(_file__) / 'app.conf')
It will fail.

Just like ipaddress, this is a deliberate design choice that avoids
coupling low level APIs to a high level convenience library.

Cheers,
Nick.

[Antoine Pitrou]

Post by Nick Coghlan

Post by Juancarlo Añez
f = open(Path(_file__) / 'app.conf')
It will fail.

Just like ipaddress, this is a deliberate design choice that avoids
coupling low level APIs to a high level convenience library.

Note the gap could be crossed without coupling by introducing a __path__
protocol (or something similar for IP addresses).

Regards

Antoine.

[Nick Coghlan]

Post by Antoine Pitrou

Post by Nick Coghlan

Post by Juancarlo Añez
f = open(Path(_file__) / 'app.conf')
It will fail.

Just like ipaddress, this is a deliberate design choice that avoids
coupling low level APIs to a high level convenience library.

Note the gap could be crossed without coupling by introducing a __path__
protocol (or something similar for IP addresses).

My main concern with that approach is the sheer number of places we'd
need to touch. I'm not implacably opposed to the idea, I just strongly
suspect it wouldn't be worth the hassle to save the str() calls, as:

- explicit str() calls would still be needed for anyone still
supporting older versions of Python
- explicit str() calls would still be needed when dealing with third
party libraries that don't support the new protocol yet
- we wouldn't get to simplify any of the low level APIs, since they'd
still need to support str objects - the new protocol would be strictly
additive.

Cheers,
Nick.

[Wolfgang Maier]

Post by Terry Reedy

Post by Wolfgang Maier

Post by Juancarlo Añez
Better yet, use the pathlib module.

Thank for the reminder. I took a better look at it.

Post by Wolfgang Maier
As it stands, the pathlib module is only provisional plus,

'Provisional' means that there *could* be a few api changes that would
break code. The module is not going away.

The 3.4 docs explicitly mention the possibility:

<quote>

Note:

This module has been included in the standard library on a provisional
basis. Backwards incompatible changes (up to and including removal of
the package) may occur if deemed necessary by the core developers.

</quote>

Post by Terry Reedy

Post by Wolfgang Maier
IMO, kind of overkill for a simple task like that.

Overkill?
import pathlib as path
import os.path as path
are equally easy
The 'simple task' combines joining, normalizing, and 'absoluting'.
pathlib.Path joins, Path.resolve normalizes and 'absolutes'. Together
they combine the functions of os.path.join, os.path.abspath and
os.path.normpath, with a nicer syntax, and with OS awareness.

Yes, the syntax is nicer *now*, but with my proposed change to

Post by Terry Reedy

Post by Wolfgang Maier

Post by Juancarlo Añez

path.Path('../../../Python27/lib', 'ast.py').resolve()

os.path as proposed:
os.path.abspath('ast.py', '../../../Python27/lib')

So I would see this as an argument for the proposal rather than against it.

Even if the pathlib module will stay, I am not sure whether that should
exclude enhancements in overlapping parts of os.path.

Anyway, that whole thing is not that important to me, so if nobody finds
it useful, then let's stick to the status quo.

[Terry Reedy]

Post by Wolfgang Maier
Yes, the syntax is nicer *now*, but with my proposed change to

path.Path('../../../Python27/lib', 'ast.py').resolve()

os.path.abspath('ast.py', '../../../Python27/lib')
So I would see this as an argument for the proposal rather than against it.
Even if the pathlib module will stay, I am not sure whether that should
exclude enhancements in overlapping parts of os.path.

I understand your reasoning. But it leaves out the following. When a
feature is added, use of the feature makes code incompatible with
previous versions. So we generally like new features to add more than
this one would.

If you look hard enough, I am sure that you can find an addition that by
this criteria should not have been added. If you do, I will probably
agree that it should not have been.

Post by Wolfgang Maier
Anyway, that whole thing is not that important to me, so if nobody finds
it useful, then let's stick to the status quo.

It is a matter of useful enough to justify the cost.

[Stephen Hansen]

This post might be inappropriate. Click to display it.

[Serhiy Storchaka]

Post by Wolfgang Maier
currently, os.path.abspath(somepath) is, essentially, equivalent to
os.path.normpath(os.path.join(os.getcwd(),path)).

Actually currently posixpath.abspath() is more complicated and
ntpath.abspath() has totally different implementation.

Post by Wolfgang Maier
os.path.normpath(os.path.join(startdir, path))
or even
os.path.normpath(os.path.join(os.path.abspath(startdir), path))

Yes, it is natural and straightforward way. You can define your own
function if you need this often.

Post by Wolfgang Maier
Before posting I checked the bug tracker and found that this idea has
been brought up years ago (http://bugs.python.org/issue9882), but not
pursued further.
The patch suggested there is a bit of an oversimplification, but I have
my own one, which I could provide if someone's interested.
For issue9882 it was suggested to bring it up on python-ideas, but to
the best of my knowledge that was never done, so I'm doing it now.
Thoughts ?

This will add to abspath() a feature which is unrelated to the purpose
of abspath(). This will complicate the API without significant benefit.

I'm -1.

[Wolfgang Maier]

Post by Serhiy Storchaka

Post by Wolfgang Maier
currently, os.path.abspath(somepath) is, essentially, equivalent to
os.path.normpath(os.path.join(os.getcwd(),path)).

Actually currently posixpath.abspath() is more complicated and
ntpath.abspath() has totally different implementation.

I know, that's why I wrote "essentially" and "equivalent" instead of "is
implemented as". It's still easy to patch both the posixpath and the
ntpath version though.

Post by Serhiy Storchaka

Post by Wolfgang Maier
os.path.normpath(os.path.join(startdir, path))
or even
os.path.normpath(os.path.join(os.path.abspath(startdir), path))

Yes, it is natural and straightforward way. You can define your own
function if you need this often.

I'm not saying, this is a must-have in Python. I don't have a problem
with sticking to normpath, just thought it's a tiny change giving some
benefit in readability.

Post by Serhiy Storchaka

Post by Wolfgang Maier
Before posting I checked the bug tracker and found that this idea has
been brought up years ago (http://bugs.python.org/issue9882), but not
pursued further.
The patch suggested there is a bit of an oversimplification, but I have
my own one, which I could provide if someone's interested.
For issue9882 it was suggested to bring it up on python-ideas, but to
the best of my knowledge that was never done, so I'm doing it now.
Thoughts ?

This will add to abspath() a feature which is unrelated to the purpose
of abspath(). This will complicate the API without significant benefit.

It would not complicate the API all that much. If you don't want to use
the argument, just ignore it, it would be optional. As pointed out in
the bug tracker issue, it is also not without precedence,
os.path.relpath has a start argument already.

[Serhiy Storchaka]

Post by Wolfgang Maier
I'm not saying, this is a must-have in Python. I don't have a problem
with sticking to normpath, just thought it's a tiny change giving some
benefit in readability.

To me explicit well known join() and normpath() are more readable then
unexpected second argument to abspath().

Post by Wolfgang Maier

Post by Serhiy Storchaka
This will add to abspath() a feature which is unrelated to the purpose
of abspath(). This will complicate the API without significant benefit.

It would not complicate the API all that much. If you don't want to use
the argument, just ignore it, it would be optional. As pointed out in
the bug tracker issue, it is also not without precedence,
os.path.relpath has a start argument already.

The inverse of two-argument relpath() is join(), not abspath().
Two-argument relpath() is only the way to compute relative patch between
two patches. This is essential functionality, there is no redundancy.
But two-argument abspath() will be redundant.

Not every one-line function should be added to the stdlib. And I found
only 10 usages of normpath(join()) combination in Python source three
(including 4 in tests and 4 in PC build script, therefore only 2 in the
stdlib itself) against 205 usages of abspath().

[Wolfgang Maier]

Post by Serhiy Storchaka

Post by Wolfgang Maier
I'm not saying, this is a must-have in Python. I don't have a problem
with sticking to normpath, just thought it's a tiny change giving some
benefit in readability.

To me explicit well known join() and normpath() are more readable then
unexpected second argument to abspath().

Ok, I just seem to think differently than all of you. whenever I need
this functionality (and just like for the stdlib, it's less often than
regular abspath), I think: oh, this must be addressable with abspath,
then after a moment I realize there is no start option like in relpath.
Then I consult the docs where I find this for abspath:

"Return a normalized absolutized version of the pathname path. On most
platforms, this is equivalent to calling the function normpath() as
follows: normpath(join(os.getcwd(), path))."

From which the solution is apparent.
Never have I thought first, ah, that's a job for normpath. Maybe that's
because I can't remember a single case where I used normpath for
anything else in my code, so I'm kind of thinking about normpath as a
low-level function needed a lot in os.path, but typically not needed
much outside of it because there are higher-level functions like abspath
that do the normalization in the background.

It's interesting to learn that I seem to be quite alone with this view,
but that's ok, I'm sure it will help me remember normpath next time :)

Post by Serhiy Storchaka
Not every one-line function should be added to the stdlib. And I found
only 10 usages of normpath(join()) combination in Python source three
(including 4 in tests and 4 in PC build script, therefore only 2 in the
stdlib itself) against 205 usages of abspath().