How do I write a setup.py for a twistd/twisted plugin that works with setuptools, distribute, etc?

By preventing pip from writing the line twisted to egg-info/top_level. Txt you can keep using packages=..., 'twisted. Plugins' and have a working pip uninstall that doesn't remove all of twisted This involves monkeypatching setuptools/distribute near the top of your setup.Py Here is a sample setup.py.

By preventing pip from writing the line "twisted" to . Egg-info/top_level. Txt, you can keep using packages=..., 'twisted.

Plugins' and have a working pip uninstall that doesn't remove all of twisted/. This involves monkeypatching setuptools/distribute near the top of your setup.py. Here is a sample setup.Py: from distutils.

Core import setup # When pip installs anything from packages, py_modules, or ext_modules that # includes a twistd plugin (which are installed to twisted/plugins/), # setuptools/distribute writes a Package. Egg-info/top_level. Txt that includes # "twisted".

If you later uninstall Package with `pip uninstall Package`, # pip removes all of twisted/ instead of just Package's twistd plugins. See # https://github.Com/pypa/pip/issues/355 # # To work around this problem, we monkeypatch # setuptools.command. Egg_info.

Write_toplevel_names to not write the line # "twisted". This fixes the behavior of `pip uninstall Package`. Note that # even with this workaround, `pip uninstall Package` still correctly uninstalls # Package's twistd plugins from twisted/plugins/, since pip also uses # Package.

Egg-info/installed-files. Txt to determine what to uninstall, # and the paths to the plugin files are indeed listed in installed-files.txt. Try: from setuptools.

Command import egg_info egg_info. Write_toplevel_names except (ImportError, AttributeError): pass else: def _top_level_package(name): return name. Split('.

', 1)0 def _hacked_write_toplevel_names(cmd, basename, filename): pkgs = dict. Fromkeys( _top_level_package(k) for k in cmd.distribution. Iter_distribution_names() if _top_level_package(k)!

= "twisted" ) cmd. Write_file("top-level names", filename, '\n'. Join(pkgs) + '\n') egg_info.

Write_toplevel_names = _hacked_write_toplevel_names setup( name='MyPackage', version='1.0', description="You can do anything with MyPackage, anything at all.", url="http://example.com/", author="John Doe", author_email="jdoe@example. Com", packages='mypackage', 'twisted. Plugins', # You may want more options here, including install_requires=, # package_data=, and classifiers= ) # Make Twisted regenerate the dropin.

Cache, if possible. This is necessary # because in a site-wide install, dropin. Cache cannot be rewritten by # normal users.

Try: from twisted. Plugin import IPlugin, getPlugins except ImportError: pass else: list(getPlugins(IPlugin)) I've tested this with pip install, pip install --user, and easy_install. With any install method, the above monkeypatch and pip uninstall work fine.

You might be wondering: do I need to clear the monkeypatch to avoid messing up the next install? (e.g. Pip install --no-deps MyPackage Twisted; you wouldn't want to affect Twisted's top_level.txt.) The answer is no; the monkeypatch does not affect another install because pip spawns a new python for each install. Related: keep in mind that in your project, you must not have a file twisted/plugins/__init__.py.

If you see this warning during installation: package init file 'twisted/plugins/__init__. Py' not found (or not a regular file) it is completely normal and you should not try to fix it by adding an __init__.py.

1 Hacktastic! The uninstall of the plugin file will only occur in this case if pip was used to install the plugin-containing project (as opposed to easy_install or "python setup. Py install"), because only pip writes installed-files.

Txt at install time. I think that's a drawback that any solution to this problem can't avoid, though, since having a full list of installed files is the only way to uninstall correctly if projects are allowed to go dropping files into other projects' packages. – Carl Meyer Sep 23 at 7:27 This could use a little more explaining to be a complete answer: what does the setup.Py look like?

How does this work with a raw setup. Py? What about with easy_install?

How about with a --user install, etc. But, as Carl put it: hacktastic!(Hopefully this proves that it is possible to fix github.Com/pypa/pip/issues/355 - at least in the case where pip was doing the installing ...) – Glyph Sep 23 at 7:49 1 Updated answer per Glyph's comment – Ivan Kozik Sep 23 at 9:20 Now: how many of these systems support a proper post-install hook that properly generates dropin. Cache as per twistedmatrix. Com/documents/11.0.0/core/howto/plugin.

Html#auto3 :) – Glyph Sep 23 at 9:25 2 Updated with dropin. Cache regeneration – Ivan Kozik Sep 23 at 9:56.

It could fail if the installation of the package doesn't put the package data into a directory which is on the sys.path. In that case the Twisted plugin loader wouldn't find it. However, all installations of Python packages that I know of will put it into the same directory where they are installing the Python modules or packages themselves, so that won't be a problem.

2 One of the problems with using package_data like that is that you still need to list 'twisted. Plugins' in the the list of packages; this results in pip uninstall blowing away your entire Twisted installation. – mithrandi Sep 2 at 9:32 Is there any way to inform pip not to do this?

Is there a bug report open in the Pip bugtracker? – Glyph Sep 4 at 18:13 1 I haven't opened a bug report, but it does seem like a bug in pip; it has a record of exactly what files it installed, so I don't see why it needs to remove other files/directories that it didn't install (with the possible caveat of . Pyco files) – mithrandi Sep 4 at 20:47 Aside from this bug in pip, are there any other issues?

It sounds like this may be the right option. – Glyph Sep 10 at 6:01 1 I've now filed a bug report here: github. Com/pypa/pip/issues/355 -- the fact that twisted ends up in top_level.

Txt when installing a plugin module like this does suggest that the this approach may not be the right thing to do. – mithrandi Sep 20 at 9:17.

Maybe you could adapt the package_data idea to use data_files instead: it wouldn’t require you to list twisted. Plugins as package, as it uses absolute paths. It would still be a kludge, though.My tests with pure distutils have told me that its is possible to overwrite files from another distribution.

I wanted to test poor man’s namespace packages using pkgutil. Extend_path and distutils, and it turns out that I can install spam/ham/__init__. Py with spam.

Ham/setup. Py and spam/eggs/__init__. Py with spam.

Eggs/setup.py. Directories are not a problem, but files will be happily overwritten. I think this is actually undefined behavior in distutils which trickles up to setuptools and pip, so pip could IMO close as wontfix.

What is the usual way to install Twisted plugins? Drop-it-here by hand?

There is no "usual" way to install Twisted plugins - there are a couple of different random ideas floating around, each with their own drawbacks. This question is part of an effort to nail down one "right" way to do it, and at least have some concept of what the drawbacks are and how to cope with them. – Glyph Sep 23 at 5:14 Pip has closed as wontfix :-) Does the data_files approach work?

If so it seems like the most promising approach listed here. It would have the same drawback as the "patch write_toplevel_names" approach: the plugin would only be uninstalled correctly if pip had been used to install it. – Carl Meyer Sep 23 at 7:24 2 Ivan tells me (but cannot comment due to SO's weird reputation limits) - One problem with this approach is that pip uninstall will not remove the corresponding .

Pyc for your . Py data_file. You can't add the .

Pyc to your data_files= either, because PYTHON_DONT_WRITE_BYTECODE might be set. – Glyph Sep 23 at 8:28 The . Pyc/.

Pyo issue is a big one. If pip supports uninstall hooks, it could be done here (distutils2 will have such hooks). If not, it only shows that you must use py_modules or packages to include Python files.

– Éric Araujo Sep 23 at 17:12 1 In the future, I hope PEP 402 is accepted, then distutils2 will add support for it and Twisted will be able to change its custom idea of plugins for a namespace package (think mercurial and hgext). – Éric Araujo Sep 23 at 17:16.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.