opensc split - how? [u]

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

opensc split - how? [u]

Andreas Jellinghaus-2
I would like to get feedback from everyone who compiles
opensc himself.

opensc 0.9.* is one big fat package. it includes everything,
many people only need to install openct+opensc and they have
all they need. (or driver + pcsc-lite + opensc).

now we want to split opensc into several smaller parts, to make
it easier to maintain. the question is, how many small parts
do we want? how big is the gain from smaller parts compared
to the overhead of compiling yet another package?

one possible direction is this:
 - opensc
 - libp11
 - pam_p11
 - pam_pkcs11
 - engine_pkcs11

each with its own web page on opensc.org, each with
its own package and svn tree and ticket sytem.

the positive part of such a split:
 - libp11 does not depend on opensc, can be used with
   any pkcs#11 library.
 - same with pam_p11 which uses libp11.
 - same with pam_pkcs11 (currently standalone).
 - same with engine_pkcs11 (will use libp11).
 - configure code was a big mess. now it will look a lot
   better as only the engine package needs to have the engine
   mess, all other can simply use pkg-config and openssl.pc.
   pam configure code only in the pam package etc.
 - each package might be easier to understand.

but of course it also has downsides:
 - getting the big picture will be a lot more difficult.
 - depending on how we move files around in svn we will loose
   the history.
 - compiling all packages in the right order is a lot more difficult.
   (well most packages are standalone / only need libp11 and openssl, so
   it shouldn't be that hard).
 - for us developers the suite of regression tests is essential.
   the current tests use opensc and tools only, but we can't grow
   the regression test suite with such a split. or we would need
   to duplicate it or change it somehow, so it can deal with the
   parts seperated.

if you havee read this far, please comment on this issue.
Doing the best thing is much easier with feedback.

Regards, Andreas
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Martin Preuss-2
Hi,

On Monday 22 August 2005 20:36, Andreas Jellinghaus [c] wrote:
[...]
> opensc 0.9.* is one big fat package. it includes everything,
> many people only need to install openct+opensc and they have
> all they need. (or driver + pcsc-lite + opensc).
[...]

This is just my personal opinion, but I think you should continue to have one
tarball and let the distributions (i.e. package managers) do the split: This
combines most of the positive effects of both approaches (split/non-split).

You can create multiple separate RPM/deb packages from a single tarball.
"Normal" users will use the packages of the distributions anyway, and users
who want to compile OpenSC themselves should be able to deal with the current
structure of having one or two tarballs...

With AqBanking/Libchipcard2 we went the other way around: We had many
complaints about users who didn't know which tarball/packages they needed to
install in order to use it properly. So we just decreased the number of
packages by incorporating them into a single tarball (those packages which
most user would need anyway).
The splitting is now done by the package manager out of a single tarball and
the programmers don't have to worry about how to divide the code into all
those packages :-) (which at least with Debian sometimes is a science of it's
own)

Just my two Ā¢ents...


Regards
Martin

--
"Things are only impossible until they're not"

AqBanking - http://www.aquamaniac.de/aqbanking/
LibChipcard - http://www.libchipcard.de/
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Marc Bevand-2
In reply to this post by Andreas Jellinghaus-2
Andreas Jellinghaus [c] wrote:
>
> one possible direction is this:
>  - opensc
>  - libp11
>  - pam_p11
>  - pam_pkcs11
>  - engine_pkcs11

IMHO such a split is too fine-grained; pam_p11, pam-pkcs11 and
engine_pkcs11 could be regrouped into a single package: openp11,
or openPXI (XI = roman for 11, I am sick of seeing 11 everywhere :P).
Leaving libp11 in a single package is fine. This way we would have:
  - openct
  - opensc
  - openpxi
  - libp11
I think this would be a good compromise.

--
 Marc Bevand                              http://epita.fr/~bevand_m
 Computer Science School EPITA - System, Network and Security Dept.
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Ludovic Rousseau
On 22/08/05, Marc Bevand <[hidden email]> wrote:
> Andreas Jellinghaus [c] wrote:
> >
> > one possible direction is this:
> >  - opensc
> >  - libp11
> >  - pam_p11
> >  - pam_pkcs11
> >  - engine_pkcs11

OK for me.

pam_p11 is the demo app [1] you (Andreas) wrote to test libp11? Do we
need to start yet another pam PKCS#11 project instead of merging the
good ideas in pam_pkcs11?

> IMHO such a split is too fine-grained; pam_p11, pam-pkcs11 and
> engine_pkcs11 could be regrouped into a single package: openp11,
> or openPXI (XI = roman for 11, I am sick of seeing 11 everywhere :P).
> Leaving libp11 in a single package is fine. This way we would have:
>   - openct
>   - opensc
>   - openpxi
>   - libp11
> I think this would be a good compromise.

I think it is important to have different projects with different
release cycles. I want to be able to release a new pam-pkcs11 without
having to wait after engine_pkcs11.

If the problem is that users will not know which tar file to fetch we
can merge different projects in one tar file. I did that for the
muscleframework [2] archive (for historical reasons).
muscleframework-1.1.5 contains CFlexPlugin-1.1.3,
libmusclepkcs11-0.0.11, MCardPlugin-1.0.3 and MusclePAM.

We can then have a big tar file with everything inside and a shell
script that will untar and build everything.

Note that it is possible to build engine_pkcs11 even if libp11 is _not
yet_ installed by giving correct arguments to engine_pkcs11'
./configure script. It is also possible to build the regression tests
without installing anything from the big tar file.

Would that be a valid solution?

Bye,

[1] http://opensc.org/pipermail/opensc-devel/2005-July/006532.html
[2] https://alioth.debian.org/project/shownotes.php?release_id=183

--
  Dr. Ludovic Rousseau
 For private mail use [hidden email] and not "big brother" Google
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Andreas Jellinghaus-2
In reply to this post by Martin Preuss-2
On Monday 22 August 2005 21:03, Martin Preuss wrote:
> This is just my personal opinion, but I think you should continue to have
one
> tarball and let the distributions (i.e. package managers) do the split: This
> combines most of the positive effects of both approaches (split/non-split).

how do you deal with compile time options?
I dislike them, because they make supporting some software a lot more
difficult. pam_pkcs11 for example contains mapper for many different
systems, so it needs a lot of libraries to compile all mappers.
the code can be changed to skip those parts where the libraries are
missing, but then support etc. is harder.

for pam_pkcs11 you would see the result in a missing mapper_*.so,
but for opensc it is more difficult: whether or not opensc was
compiled with openssl support is difficult to see. or whether that
openssl was a version with engine, without engine, with engine but
buggy, or with engine and fixed. and of course our bug detection
and workaround could be wrong, so that is an extra option we need
to know.

the good thing about libp11 is: it works with other pkcs#11 libs, too.
so it might be good to be able to compile it without opensc, right?

> With AqBanking/Libchipcard2 we went the other way around: We had many
> complaints about users who didn't know which tarball/packages they needed to
> install in order to use it properly. So we just decreased the number of
> packages by incorporating them into a single tarball (those packages which
> most user would need anyway).

do you think it would be acceptable to ship a shell script, that
wget's and compiles each tar file in the right order? I think kde is
doing something like that.

I have similar fears that many packages will be disliked. however two
simple packages you can install or skip as you want might be easier
than a combined package with dependencies for the part you don't want?
a dependency for some feature you don't want annoys people a lot, I guess.

what do you think about these issues?
thanks for your comment!

Regards, Andreas
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Andreas Jellinghaus-2
In reply to this post by Marc Bevand-2
On Monday 22 August 2005 21:23, Marc Bevand wrote:
> IMHO such a split is too fine-grained; pam_p11, pam-pkcs11 and
> engine_pkcs11 could be regrouped into a single package: openp11,
> or openPXI (XI = roman for 11, I am sick of seeing 11 everywhere :P).
> Leaving libp11 in a single package is fine. This way we would have:
>   - openct
>   - opensc
>   - openpxi
>   - libp11
> I think this would be a good compromise.

Hi Marc,

the problem I see is dependencies.
pam_pkcs11 has mappers for many systems like kerberos.
so any package that includes it would need all those
dependencies. and I guess the real big problem when compiling
is not configure / make / make install, but all those compile
options and dependencies.

also engine_pkcs11 would have that complexity with openssl:
which version is available? if 0.9.6: with or without engine support?
if 0.9.7: libcrypto.so bug/feature fixed (0.9.7d+) or nor?
and if 0.9.7 and bug, then it would use the old workaround.
such situations end in nasty configure code. and nasty code attracts
problems. so someone who doesn't want or need that engine should
not be bothered with it?

that's why I'm thinking about splitting libp11 / pam_p11 / pam_pkcs11 /
engine_pkcs11, even though each part is small. but I'm not sure if
the benefit I see outwights the drawbacks of having many packages.
what do you think?

thanks for your coments!

Regards, Andreas
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Andreas Jellinghaus-2
In reply to this post by Ludovic Rousseau
On Tuesday 23 August 2005 08:50, Ludovic Rousseau wrote:

> On 22/08/05, Marc Bevand <[hidden email]> wrote:
> > Andreas Jellinghaus [c] wrote:
> > >
> > > one possible direction is this:
> > >  - opensc
> > >  - libp11
> > >  - pam_p11
> > >  - pam_pkcs11
> > >  - engine_pkcs11
>
> OK for me.

thanks.

> pam_p11 is the demo app [1] you (Andreas) wrote to test libp11? Do we
> need to start yet another pam PKCS#11 project instead of merging the
> good ideas in pam_pkcs11?

pam_p11 has some few improvements that can be merged into pam_pkcs11.
the other direction is harder: pam_p11 is lightwight - no config file,
no scconf, no scdl or ltdl or any loading of modules. also no crl
or ocsp checks. compile dependencies: only pam, p11 and openssl.
pam_pkcs11 is heavywight on all those issues, and thus harder to
use, compile, understand etc. so I'm still undecided, keeping
it around as lightwight module might not be so bad.

> I think it is important to have different projects with different
> release cycles. I want to be able to release a new pam-pkcs11 without
> having to wait after engine_pkcs11.
good point. opensc release cycle is horrible, maybe a smaller package
can be updated more often.

> If the problem is that users will not know which tar file to fetch we
> can merge different projects in one tar file. I did that for the
> muscleframework [2] archive (for historical reasons).
> muscleframework-1.1.5 contains CFlexPlugin-1.1.3,
> libmusclepkcs11-0.0.11, MCardPlugin-1.0.3 and MusclePAM.
>
> We can then have a big tar file with everything inside and a shell
> script that will untar and build everything.

good idea. or maybe only a shell script that wget's each file, unpacks
them, and compiles and installs them in the right order?

> Note that it is possible to build engine_pkcs11 even if libp11 is _not
> yet_ installed by giving correct arguments to engine_pkcs11'
> ./configure script.

hu? engine_pkcs11 uses structures defined in libp11.h, so how could
it be build without? maybe I'm confused?

> It is also possible to build the regression tests
> without installing anything from the big tar file.

for now regression tests use opensc functionality only.
also they use the code in the source tree. so you can
test a new version without "make install". we could
drop the later and require some applications to be
installed (or simply skip some tests if the other
app or library is missing).

so what do you think, how should we proceed?
thanks for your comments.

Regards, Andreas
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Marc Bevand-2
In reply to this post by Andreas Jellinghaus-2
Andreas Jellinghaus [c] wrote:
|
| the problem I see is dependencies.
| [...]
| that's why I'm thinking about splitting libp11 / pam_p11 / pam_pkcs11 /
| engine_pkcs11, even though each part is small. but I'm not sure if
| the benefit I see outwights the drawbacks of having many packages.
| what do you think?

After reconsideration, as you describe it, I think you are right by
choosing to split opensc this way.

Most end-users should not be confused because it is the job of the
distro packages maintainers to work out the dependencies.

--
 Marc Bevand                              http://epita.fr/~bevand_m
 Computer Science School EPITA - System, Network and Security Dept.
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Ludovic Rousseau
In reply to this post by Andreas Jellinghaus-2
On 23/08/05, Andreas Jellinghaus [c] <[hidden email]> wrote:

> On Tuesday 23 August 2005 08:50, Ludovic Rousseau wrote:
> > pam_p11 is the demo app [1] you (Andreas) wrote to test libp11? Do we
> > need to start yet another pam PKCS#11 project instead of merging the
> > good ideas in pam_pkcs11?
>
> pam_p11 has some few improvements that can be merged into pam_pkcs11.
> the other direction is harder: pam_p11 is lightwight - no config file,
> no scconf, no scdl or ltdl or any loading of modules. also no crl
> or ocsp checks. compile dependencies: only pam, p11 and openssl.
> pam_pkcs11 is heavywight on all those issues, and thus harder to
> use, compile, understand etc. so I'm still undecided, keeping
> it around as lightwight module might not be so bad.

I see your point. But I may not agree :-)

> > We can then have a big tar file with everything inside and a shell
> > script that will untar and build everything.
>
> good idea. or maybe only a shell script that wget's each file, unpacks
> them, and compiles and installs them in the right order?

You may not have Internet access at compile time.
But you can provide this added feature if needed.

> > Note that it is possible to build engine_pkcs11 even if libp11 is _not
> > yet_ installed by giving correct arguments to engine_pkcs11'
> > ./configure script.
>
> hu? engine_pkcs11 uses structures defined in libp11.h, so how could
> it be build without? maybe I'm confused?

Just use the correct CFLAGS=... LIBS=... arguments to ./configure

> so what do you think, how should we proceed?
> thanks for your comments.

What I would like now:
- Create a trac project for libp11
- move libp11 source code outside of opensc
- make a release of libp11

- create a trac project for sslengines?
- move sslengines source code outside of opensc

- work on a build script to automate things

Bye,

--
 Dr. Ludovic Rousseau
 For private mail use [hidden email] and not "big brother" Google
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Andreas Jellinghaus-2
In reply to this post by Andreas Jellinghaus-2
Thanks to everone for your comments.

I will now add libp11, engine_pkcs11 and pam_p11 trac and svn
setups. pam_p11 might be merged with pam_pkcs11 and thus removed
someday, not sure.

Once those new packages work fine and we have a first package
released I will remove the code from opensc.

Then both engines will be removed from opensc, but only pkcs11
engine will make it in a new package. That includes engine_opensc
will be lost. I found yet another bug/feature of it: you can't
sign two certificates with openssl ca command with it. The
opensc engine has a lot of limitations, so I don't think it is
a loss. Let me know if you think different, we could package
it somehow.

Regards, Andreas
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Martin Paljak
On 8/24/05, Andreas Jellinghaus [c] <[hidden email]> wrote:
> Thanks to everone for your comments.
>
> I will now add libp11, engine_pkcs11 and pam_p11 trac and svn
> setups. pam_p11 might be merged with pam_pkcs11 and thus removed
> someday, not sure.
Hmm, Sorry for a late reply - i was in the woods for a small vacation.

All sounds good except one thing: I would not split the things into
totally different trac instaces. I think trac works well for a
'project'. Lets call this project opensc. With different components.
It just makes it a bit harder to have a nice and linked wiki setup for
example. It is hard to maintain a nice ticket system with different
release cycles with such setup. Those components (libp11 and engines)
are useful separately but they still make sense if used together.

Though it does help to make a clear separation in functionality if
they remain separated.


m.
--
Martin Paljak
[hidden email]
http://martin.paljak.pri.ee/
+372.5156495 - phone
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel
Reply | Threaded
Open this post in threaded view
|

Re: opensc split - how? [u]

Andreas Jellinghaus-2
On Thursday 25 August 2005 19:12, Martin Paljak wrote:
> All sounds good except one thing: I would not split the things into
> totally different trac instaces. I think trac works well for a
> 'project'. Lets call this project opensc. With different components.

hmm, that would mean a common svn tree, too? (So the browser still
links to all source, and commit mails can close bugs etc.)

too late for now, but if you want to work on that, we could create
a testing trac and svn (or simply use test/) and if that works
is feasible and it looks nice, we could migrate later.

but such a change might be a lot of work and there could be more
important things? it is not on my agenda so far.

> It just makes it a bit harder to have a nice and linked wiki setup for
> example. It is hard to maintain a nice ticket system with different
> release cycles with such setup.
yes, there is only one version list, not one per component. So trac
might not be designed for such installations.

> Those components (libp11 and engines)
> are useful separately but they still make sense if used together.

engine_pkcs11 will have that long and ugly openssl detection and
workaround code, so I think it is nicer to not have that ballast
in libp11 configure code.

Regards, Andreas
_______________________________________________
opensc-devel mailing list
[hidden email]
http://www.opensc.org/cgi-bin/mailman/listinfo/opensc-devel