Discussion:
​Re: PSPP CTABLES
Michał Dubrawski
2018-07-26 07:56:17 UTC
Permalink
Hi Ben,

it is great to hear that you are working with spv output. I know that they
were changing it from version to version so it seem like quite a challenge,
but based on what challenges you have faced before when making PSPP
compatible to SPSS I'm sure you will handle it :)

CTABLES or Tables in the future will be very useful but I think that what
would be most important for students, social sicentists, psychologists,
market researchers or virtualy anyone who analyse any survey results would
be the possibility to display mrsets in one table, which could be done in
SPSS with MULT RESPONSE command. Without it PSPP wouldn't be an option to
display survey results because almost all surveys have multi response
questions. The only free alternative would be some packages in R which are
not as accessible (easy to use) as PSPP.

Any way to display MRSETS in the PSPP output would be very valuable for a
lot of people even if it would be a cross tabs extension allowing the usage
of mrsets in it or something like that. When people graduate from
psychology, sociology, social science or pedagogics departments they have
knowledge how to build a surveys - there are a lot of tools that allow them
to collect data via online surveys, but when it comes to displaying the
results in tables for the reports they all have a software problem because
many non-profits and startups cannot afford SPSS + Custom Tables licenses,
and from my experience that is exactly the software they know how to use
because of their Universities.

That said, thanks again to Ben and all the developers and all other people
involved with PSPP project - you have done tremendous work so far.

Michał Dubrawski,
Date: Wed, 25 Jul 2018 08:48:33 -0700
​​
Re: PSPP CTABLES
Content-Type: text/plain; charset=utf-8
I think it's likely that I'll change PSPP to use the same output model.
One reason that it differed until now is simply that it wasn't clear to
me how SPSS really modeled output. The documentation on it is pretty
muddy.
I've now got code that reads and understands most .spv files just fine.
I'm still working on reading the oldest kind of .spv files, which use a
really funny way to encode tables.
FWIW, I'm excited that you're working on reading SPV format. I had
assumed that the implicit output model might preclude compatibility.
-Alan
There might be some small hope here because I've been spending a lot of
time lately working to make PSPP understand SPSS's .spv output file
format, so that it can read (and write?) it directly. Implicit in that
format is an understanding of the SPSS output model, which seems to be
entangled quite a bit with the CTABLES syntax. It might therefore lead
someday to CTABLES implementation (but not soon).
I asked about this a few years ago. I gather it would be difficult.
Mind you, I used TABLES in SPSS in about 1990 and got to quite like it and
used it a lot. So, here we are in 2018 and I use CTABLES in SPSS, and can?t
get the hang of it, at least not much more than the basic usage of it.
Can?t stand it. I long for the old TABLES function. Not available in SPSS.
So, here?s the thing: perhaps PSPP should develop its own version of TABLES
with just the basics, without trying to emulate SPSS because they have
stuffed up something that used to be quite good.
From: Pspp-users [mailto:pspp-users-bounces+ronald.crichton=
Sent: Tuesday, 24 July 2018 6:29 PM
Subject: PSPP CTABLES
Hi, I would like to ask if there is any progression in CTABLES
implementation to PSPP. I read some threads (http://pspp-users.gnu.
narkive.com/r32JwdwI/what-features-of-custom-tables-
ctables-are-most-important) but they seems dead. Any suggestion?
Kind regards
Jacob
------------------------------------------------------------
-----------
This email, and any attachments, may be confidential and also
privileged. If you are not the intended recipient, please notify the sender
and delete all copies of this transmission along with any attachments
immediately. You should not copy or use it for any purpose, nor disclose
its contents to any other person.
------------------------------------------------------------
-----------
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
--
Alan D. Mead, Ph.D.
President, Talent Algorithms Inc.
science + technology = better workers
http://www.alanmead.org
"You're an interesting species. An interesting mix.
You're capable of such beautiful dreams, and such
horrible nightmares. You feel so lost, so cut off,
so alone, only you're not. See, in all our
searching, the only thing we've found that makes
the emptiness bearable, is each other."
-- Carl Sagan, Contact
Ben Pfaff
2018-07-26 16:50:27 UTC
Permalink
MULT RESPONSE might also benefit from the work I'm doing. I've looked
at MULT RESPONSE before, and one of the things that kept me from
implementing it is the output complexity. CROSSTABS also has very
complex output routines, to the extent the complexity of the output
comes close to overshadowing the complexity of the data analysis. MULT
RESPONSE is worse.

But most of the complexity comes from how PSPP internally has no
systematic approach to displaying data as tables. (It does have decent
support for rendering tables into different formats, but not for
converting data into tables to be rendered.) The SPV format actually
points a good way forward on this point, so when the appropriate amount
of work is done, it should allow CROSSTABS to be nicer internally and
then a reasonable MULT RESPONSE implementation too.
Post by Michał Dubrawski
Hi Ben,
it is great to hear that you are working with spv output. I know that they
were changing it from version to version so it seem like quite a challenge,
but based on what challenges you have faced before when making PSPP
compatible to SPSS I'm sure you will handle it :)
CTABLES or Tables in the future will be very useful but I think that what
would be most important for students, social sicentists, psychologists,
market researchers or virtualy anyone who analyse any survey results would
be the possibility to display mrsets in one table, which could be done in
SPSS with MULT RESPONSE command. Without it PSPP wouldn't be an option to
display survey results because almost all surveys have multi response
questions. The only free alternative would be some packages in R which are
not as accessible (easy to use) as PSPP.
Any way to display MRSETS in the PSPP output would be very valuable for a
lot of people even if it would be a cross tabs extension allowing the usage
of mrsets in it or something like that. When people graduate from
psychology, sociology, social science or pedagogics departments they have
knowledge how to build a surveys - there are a lot of tools that allow them
to collect data via online surveys, but when it comes to displaying the
results in tables for the reports they all have a software problem because
many non-profits and startups cannot afford SPSS + Custom Tables licenses,
and from my experience that is exactly the software they know how to use
because of their Universities.
That said, thanks again to Ben and all the developers and all other people
involved with PSPP project - you have done tremendous work so far.
Michał Dubrawski,
Date: Wed, 25 Jul 2018 08:48:33 -0700
​​
Re: PSPP CTABLES
Content-Type: text/plain; charset=utf-8
I think it's likely that I'll change PSPP to use the same output model.
One reason that it differed until now is simply that it wasn't clear to
me how SPSS really modeled output. The documentation on it is pretty
muddy.
I've now got code that reads and understands most .spv files just fine.
I'm still working on reading the oldest kind of .spv files, which use a
really funny way to encode tables.
FWIW, I'm excited that you're working on reading SPV format. I had
assumed that the implicit output model might preclude compatibility.
-Alan
There might be some small hope here because I've been spending a lot of
time lately working to make PSPP understand SPSS's .spv output file
format, so that it can read (and write?) it directly. Implicit in that
format is an understanding of the SPSS output model, which seems to be
entangled quite a bit with the CTABLES syntax. It might therefore lead
someday to CTABLES implementation (but not soon).
I asked about this a few years ago. I gather it would be difficult.
Mind you, I used TABLES in SPSS in about 1990 and got to quite like it and
used it a lot. So, here we are in 2018 and I use CTABLES in SPSS, and can?t
get the hang of it, at least not much more than the basic usage of it.
Can?t stand it. I long for the old TABLES function. Not available in SPSS.
So, here?s the thing: perhaps PSPP should develop its own version of TABLES
with just the basics, without trying to emulate SPSS because they have
stuffed up something that used to be quite good.
From: Pspp-users [mailto:pspp-users-bounces+ronald.crichton=
Sent: Tuesday, 24 July 2018 6:29 PM
Subject: PSPP CTABLES
Hi, I would like to ask if there is any progression in CTABLES
implementation to PSPP. I read some threads (http://pspp-users.gnu.
narkive.com/r32JwdwI/what-features-of-custom-tables-
ctables-are-most-important) but they seems dead. Any suggestion?
Kind regards
Jacob
------------------------------------------------------------
-----------
This email, and any attachments, may be confidential and also
privileged. If you are not the intended recipient, please notify the sender
and delete all copies of this transmission along with any attachments
immediately. You should not copy or use it for any purpose, nor disclose
its contents to any other person.
------------------------------------------------------------
-----------
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
--
Alan D. Mead, Ph.D.
President, Talent Algorithms Inc.
science + technology = better workers
http://www.alanmead.org
"You're an interesting species. An interesting mix.
You're capable of such beautiful dreams, and such
horrible nightmares. You feel so lost, so cut off,
so alone, only you're not. See, in all our
searching, the only thing we've found that makes
the emptiness bearable, is each other."
-- Carl Sagan, Contact
Michał Dubrawski
2018-07-26 17:24:18 UTC
Permalink
Thank you Ben. I understand the context now. Good to hear that spv format
will help. By the way does it mean that unlike SPSS which if I remember
well cannot open spv created in older versions of SPSS, PSPP would be able
to open any spv? Would it be only able to read spv with output from
procedures that are also implemented in PSPP?
One more thing I really hate how they made graphics commends which could
not be called from macros - please don't go that way if you would like to
implement it in PSPP someday :)
Post by Ben Pfaff
MULT RESPONSE might also benefit from the work I'm doing. I've looked
at MULT RESPONSE before, and one of the things that kept me from
implementing it is the output complexity. CROSSTABS also has very
complex output routines, to the extent the complexity of the output
comes close to overshadowing the complexity of the data analysis. MULT
RESPONSE is worse.
But most of the complexity comes from how PSPP internally has no
systematic approach to displaying data as tables. (It does have decent
support for rendering tables into different formats, but not for
converting data into tables to be rendered.) The SPV format actually
points a good way forward on this point, so when the appropriate amount
of work is done, it should allow CROSSTABS to be nicer internally and
then a reasonable MULT RESPONSE implementation too.
Post by Michał Dubrawski
Hi Ben,
it is great to hear that you are working with spv output. I know that
they
Post by Michał Dubrawski
were changing it from version to version so it seem like quite a
challenge,
Post by Michał Dubrawski
but based on what challenges you have faced before when making PSPP
compatible to SPSS I'm sure you will handle it :)
CTABLES or Tables in the future will be very useful but I think that what
would be most important for students, social sicentists, psychologists,
market researchers or virtualy anyone who analyse any survey results
would
Post by Michał Dubrawski
be the possibility to display mrsets in one table, which could be done in
SPSS with MULT RESPONSE command. Without it PSPP wouldn't be an option
to
Post by Michał Dubrawski
display survey results because almost all surveys have multi response
questions. The only free alternative would be some packages in R which
are
Post by Michał Dubrawski
not as accessible (easy to use) as PSPP.
Any way to display MRSETS in the PSPP output would be very valuable for a
lot of people even if it would be a cross tabs extension allowing the
usage
Post by Michał Dubrawski
of mrsets in it or something like that. When people graduate from
psychology, sociology, social science or pedagogics departments they have
knowledge how to build a surveys - there are a lot of tools that allow
them
Post by Michał Dubrawski
to collect data via online surveys, but when it comes to displaying the
results in tables for the reports they all have a software problem
because
Post by Michał Dubrawski
many non-profits and startups cannot afford SPSS + Custom Tables
licenses,
Post by Michał Dubrawski
and from my experience that is exactly the software they know how to use
because of their Universities.
That said, thanks again to Ben and all the developers and all other
people
Post by Michał Dubrawski
involved with PSPP project - you have done tremendous work so far.
Michał Dubrawski,
Date: Wed, 25 Jul 2018 08:48:33 -0700
​​
Re: PSPP CTABLES
Content-Type: text/plain; charset=utf-8
I think it's likely that I'll change PSPP to use the same output model.
One reason that it differed until now is simply that it wasn't clear to
me how SPSS really modeled output. The documentation on it is pretty
muddy.
I've now got code that reads and understands most .spv files just fine.
I'm still working on reading the oldest kind of .spv files, which use a
really funny way to encode tables.
FWIW, I'm excited that you're working on reading SPV format. I had
assumed that the implicit output model might preclude compatibility.
-Alan
There might be some small hope here because I've been spending a
lot of
Post by Michał Dubrawski
time lately working to make PSPP understand SPSS's .spv output file
format, so that it can read (and write?) it directly. Implicit in
that
Post by Michał Dubrawski
format is an understanding of the SPSS output model, which seems
to be
Post by Michał Dubrawski
entangled quite a bit with the CTABLES syntax. It might therefore
lead
Post by Michał Dubrawski
someday to CTABLES implementation (but not soon).
I asked about this a few years ago. I gather it would be
difficult.
Post by Michał Dubrawski
Mind you, I used TABLES in SPSS in about 1990 and got to quite like it
and
Post by Michał Dubrawski
used it a lot. So, here we are in 2018 and I use CTABLES in SPSS, and
can?t
Post by Michał Dubrawski
get the hang of it, at least not much more than the basic usage of it.
Can?t stand it. I long for the old TABLES function. Not available in
SPSS.
Post by Michał Dubrawski
So, here?s the thing: perhaps PSPP should develop its own version of
TABLES
Post by Michał Dubrawski
with just the basics, without trying to emulate SPSS because they have
stuffed up something that used to be quite good.
From: Pspp-users [mailto:pspp-users-bounces+ronald.crichton=
Sent: Tuesday, 24 July 2018 6:29 PM
Subject: PSPP CTABLES
Hi, I would like to ask if there is any progression in CTABLES
implementation to PSPP. I read some threads (http://pspp-users.gnu.
narkive.com/r32JwdwI/what-features-of-custom-tables-
ctables-are-most-important) but they seems dead. Any suggestion?
Kind regards
Jacob
------------------------------------------------------------
-----------
This email, and any attachments, may be confidential and also
privileged. If you are not the intended recipient, please notify the
sender
Post by Michał Dubrawski
and delete all copies of this transmission along with any attachments
immediately. You should not copy or use it for any purpose, nor
disclose
Post by Michał Dubrawski
its contents to any other person.
------------------------------------------------------------
-----------
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
--
Alan D. Mead, Ph.D.
President, Talent Algorithms Inc.
science + technology = better workers
http://www.alanmead.org
"You're an interesting species. An interesting mix.
You're capable of such beautiful dreams, and such
horrible nightmares. You feel so lost, so cut off,
so alone, only you're not. See, in all our
searching, the only thing we've found that makes
the emptiness bearable, is each other."
-- Carl Sagan, Contact
Ben Pfaff
2018-07-26 17:35:56 UTC
Permalink
There seem to be several SPSS output formats.

First, there's something really old (?) with an .spw extension. I
haven't looked at those at all and I don't have a corpus of them to
learn from, so PSPP won't support .spw files until those change.

Second, there are effectively two different subformats of .spv files.
Both of them wrap XML and binary members inside a .zip file. In the
older format, graphs and tables are both represented by ViZML XML. This
is a fairly awful way to represent a table, so the newer format switches
to a binary format for tables (SPSS calls this format "light"). There
are different sub-versions for that binary format, too. Currently my
reader understands the light table format (all versions I've run into)
and I'm working on the older format.

I haven't tried to make PSPP understand SPSS graphs. That's a lot more
work.

When this feature gets in, it's probably going to take several
iterations of bug reports and fixes, because I'm implementing it by
reading lots of .spv files and making sure that the reader supports
them. Even though it successfully reads the thousands of .spv files
I've collected, I imagine that there will be lots that it initially
cannot. For those, I hope that users will send them to me so that I can
add them to my corpus and fix the bugs. Other than those bugs, PSPP
should be able to open any .spv file, regardless of what procedures they
use.

Which graphics commands can't be called from macros?

Thanks,

Ben.
Post by Michał Dubrawski
Thank you Ben. I understand the context now. Good to hear that spv format
will help. By the way does it mean that unlike SPSS which if I remember
well cannot open spv created in older versions of SPSS, PSPP would be able
to open any spv? Would it be only able to read spv with output from
procedures that are also implemented in PSPP?
One more thing I really hate how they made graphics commends which could
not be called from macros - please don't go that way if you would like to
implement it in PSPP someday :)
Post by Ben Pfaff
MULT RESPONSE might also benefit from the work I'm doing. I've looked
at MULT RESPONSE before, and one of the things that kept me from
implementing it is the output complexity. CROSSTABS also has very
complex output routines, to the extent the complexity of the output
comes close to overshadowing the complexity of the data analysis. MULT
RESPONSE is worse.
But most of the complexity comes from how PSPP internally has no
systematic approach to displaying data as tables. (It does have decent
support for rendering tables into different formats, but not for
converting data into tables to be rendered.) The SPV format actually
points a good way forward on this point, so when the appropriate amount
of work is done, it should allow CROSSTABS to be nicer internally and
then a reasonable MULT RESPONSE implementation too.
Post by Michał Dubrawski
Hi Ben,
it is great to hear that you are working with spv output. I know that
they
Post by Michał Dubrawski
were changing it from version to version so it seem like quite a
challenge,
Post by Michał Dubrawski
but based on what challenges you have faced before when making PSPP
compatible to SPSS I'm sure you will handle it :)
CTABLES or Tables in the future will be very useful but I think that what
would be most important for students, social sicentists, psychologists,
market researchers or virtualy anyone who analyse any survey results
would
Post by Michał Dubrawski
be the possibility to display mrsets in one table, which could be done in
SPSS with MULT RESPONSE command. Without it PSPP wouldn't be an option
to
Post by Michał Dubrawski
display survey results because almost all surveys have multi response
questions. The only free alternative would be some packages in R which
are
Post by Michał Dubrawski
not as accessible (easy to use) as PSPP.
Any way to display MRSETS in the PSPP output would be very valuable for a
lot of people even if it would be a cross tabs extension allowing the
usage
Post by Michał Dubrawski
of mrsets in it or something like that. When people graduate from
psychology, sociology, social science or pedagogics departments they have
knowledge how to build a surveys - there are a lot of tools that allow
them
Post by Michał Dubrawski
to collect data via online surveys, but when it comes to displaying the
results in tables for the reports they all have a software problem
because
Post by Michał Dubrawski
many non-profits and startups cannot afford SPSS + Custom Tables
licenses,
Post by Michał Dubrawski
and from my experience that is exactly the software they know how to use
because of their Universities.
That said, thanks again to Ben and all the developers and all other
people
Post by Michał Dubrawski
involved with PSPP project - you have done tremendous work so far.
Michał Dubrawski,
Date: Wed, 25 Jul 2018 08:48:33 -0700
​​
Re: PSPP CTABLES
Content-Type: text/plain; charset=utf-8
I think it's likely that I'll change PSPP to use the same output model.
One reason that it differed until now is simply that it wasn't clear to
me how SPSS really modeled output. The documentation on it is pretty
muddy.
I've now got code that reads and understands most .spv files just fine.
I'm still working on reading the oldest kind of .spv files, which use a
really funny way to encode tables.
FWIW, I'm excited that you're working on reading SPV format. I had
assumed that the implicit output model might preclude compatibility.
-Alan
There might be some small hope here because I've been spending a
lot of
Post by Michał Dubrawski
time lately working to make PSPP understand SPSS's .spv output file
format, so that it can read (and write?) it directly. Implicit in
that
Post by Michał Dubrawski
format is an understanding of the SPSS output model, which seems
to be
Post by Michał Dubrawski
entangled quite a bit with the CTABLES syntax. It might therefore
lead
Post by Michał Dubrawski
someday to CTABLES implementation (but not soon).
I asked about this a few years ago. I gather it would be
difficult.
Post by Michał Dubrawski
Mind you, I used TABLES in SPSS in about 1990 and got to quite like it
and
Post by Michał Dubrawski
used it a lot. So, here we are in 2018 and I use CTABLES in SPSS, and
can?t
Post by Michał Dubrawski
get the hang of it, at least not much more than the basic usage of it.
Can?t stand it. I long for the old TABLES function. Not available in
SPSS.
Post by Michał Dubrawski
So, here?s the thing: perhaps PSPP should develop its own version of
TABLES
Post by Michał Dubrawski
with just the basics, without trying to emulate SPSS because they have
stuffed up something that used to be quite good.
From: Pspp-users [mailto:pspp-users-bounces+ronald.crichton=
Sent: Tuesday, 24 July 2018 6:29 PM
Subject: PSPP CTABLES
Hi, I would like to ask if there is any progression in CTABLES
implementation to PSPP. I read some threads (http://pspp-users.gnu.
narkive.com/r32JwdwI/what-features-of-custom-tables-
ctables-are-most-important) but they seems dead. Any suggestion?
Kind regards
Jacob
------------------------------------------------------------
-----------
This email, and any attachments, may be confidential and also
privileged. If you are not the intended recipient, please notify the
sender
Post by Michał Dubrawski
and delete all copies of this transmission along with any attachments
immediately. You should not copy or use it for any purpose, nor
disclose
Post by Michał Dubrawski
its contents to any other person.
------------------------------------------------------------
-----------
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
--
Alan D. Mead, Ph.D.
President, Talent Algorithms Inc.
science + technology = better workers
http://www.alanmead.org
"You're an interesting species. An interesting mix.
You're capable of such beautiful dreams, and such
horrible nightmares. You feel so lost, so cut off,
so alone, only you're not. See, in all our
searching, the only thing we've found that makes
the emptiness bearable, is each other."
-- Carl Sagan, Contact
Michał Dubrawski
2018-07-26 23:00:45 UTC
Permalink
Thank you Ben,

I remember the old output format from my first contact with SPSS as a
student - I found information that it was .spo file extension while .spw is
file extension for web reports:
https://stats.idre.ucla.edu/spss/seminars/introduction-to-spss-syntax-2/
" SPSS output files have the file extension .spv. You can also save output
files with the .spw extension, which is for the SPSS Web Reports format.
Older SPSS output files may have the extension .spo. These files can be
read with the SPSS Smart Viewer, which can be downloaded for free from the
IBM SPSS website. Note that only .spo files that were created using a
Windows machine can be read with the SPSS Smart Viewer. "

About graphics I meant GPL graphics:
https://www.ibm.com/support/knowledgecenter/en/SSLVMB_24.0.0/spss/base/syn_begin_gpl_overview.html

"GPL blocks cannot be contained within DEFINE-!ENDDEFINE macro definitions"

I don't really like that GPL command language, but I don't know if you ever
plan to have graphics like that in PSPP.


warm regards,
Michal
Post by Ben Pfaff
There seem to be several SPSS output formats.
First, there's something really old (?) with an .spw extension. I
haven't looked at those at all and I don't have a corpus of them to
learn from, so PSPP won't support .spw files until those change.
Second, there are effectively two different subformats of .spv files.
Both of them wrap XML and binary members inside a .zip file. In the
older format, graphs and tables are both represented by ViZML XML. This
is a fairly awful way to represent a table, so the newer format switches
to a binary format for tables (SPSS calls this format "light"). There
are different sub-versions for that binary format, too. Currently my
reader understands the light table format (all versions I've run into)
and I'm working on the older format.
I haven't tried to make PSPP understand SPSS graphs. That's a lot more
work.
When this feature gets in, it's probably going to take several
iterations of bug reports and fixes, because I'm implementing it by
reading lots of .spv files and making sure that the reader supports
them. Even though it successfully reads the thousands of .spv files
I've collected, I imagine that there will be lots that it initially
cannot. For those, I hope that users will send them to me so that I can
add them to my corpus and fix the bugs. Other than those bugs, PSPP
should be able to open any .spv file, regardless of what procedures they
use.
Which graphics commands can't be called from macros?
Thanks,
Ben.
Post by Michał Dubrawski
Thank you Ben. I understand the context now. Good to hear that spv format
will help. By the way does it mean that unlike SPSS which if I remember
well cannot open spv created in older versions of SPSS, PSPP would be
able
Post by Michał Dubrawski
to open any spv? Would it be only able to read spv with output from
procedures that are also implemented in PSPP?
One more thing I really hate how they made graphics commends which could
not be called from macros - please don't go that way if you would like to
implement it in PSPP someday :)
Post by Ben Pfaff
MULT RESPONSE might also benefit from the work I'm doing. I've looked
at MULT RESPONSE before, and one of the things that kept me from
implementing it is the output complexity. CROSSTABS also has very
complex output routines, to the extent the complexity of the output
comes close to overshadowing the complexity of the data analysis. MULT
RESPONSE is worse.
But most of the complexity comes from how PSPP internally has no
systematic approach to displaying data as tables. (It does have decent
support for rendering tables into different formats, but not for
converting data into tables to be rendered.) The SPV format actually
points a good way forward on this point, so when the appropriate amount
of work is done, it should allow CROSSTABS to be nicer internally and
then a reasonable MULT RESPONSE implementation too.
Post by Michał Dubrawski
Hi Ben,
it is great to hear that you are working with spv output. I know that
they
Post by Michał Dubrawski
were changing it from version to version so it seem like quite a
challenge,
Post by Michał Dubrawski
but based on what challenges you have faced before when making PSPP
compatible to SPSS I'm sure you will handle it :)
CTABLES or Tables in the future will be very useful but I think that
what
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
would be most important for students, social sicentists,
psychologists,
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
market researchers or virtualy anyone who analyse any survey results
would
Post by Michał Dubrawski
be the possibility to display mrsets in one table, which could be
done in
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
SPSS with MULT RESPONSE command. Without it PSPP wouldn't be an
option
Post by Michał Dubrawski
Post by Ben Pfaff
to
Post by Michał Dubrawski
display survey results because almost all surveys have multi response
questions. The only free alternative would be some packages in R
which
Post by Michał Dubrawski
Post by Ben Pfaff
are
Post by Michał Dubrawski
not as accessible (easy to use) as PSPP.
Any way to display MRSETS in the PSPP output would be very valuable
for a
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
lot of people even if it would be a cross tabs extension allowing the
usage
Post by Michał Dubrawski
of mrsets in it or something like that. When people graduate from
psychology, sociology, social science or pedagogics departments they
have
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
knowledge how to build a surveys - there are a lot of tools that
allow
Post by Michał Dubrawski
Post by Ben Pfaff
them
Post by Michał Dubrawski
to collect data via online surveys, but when it comes to displaying
the
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
results in tables for the reports they all have a software problem
because
Post by Michał Dubrawski
many non-profits and startups cannot afford SPSS + Custom Tables
licenses,
Post by Michał Dubrawski
and from my experience that is exactly the software they know how to
use
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
because of their Universities.
That said, thanks again to Ben and all the developers and all other
people
Post by Michał Dubrawski
involved with PSPP project - you have done tremendous work so far.
Michał Dubrawski,
Date: Wed, 25 Jul 2018 08:48:33 -0700
​​
Re: PSPP CTABLES
Content-Type: text/plain; charset=utf-8
I think it's likely that I'll change PSPP to use the same output
model.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
One reason that it differed until now is simply that it wasn't
clear to
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
me how SPSS really modeled output. The documentation on it is
pretty
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
muddy.
I've now got code that reads and understands most .spv files just
fine.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
I'm still working on reading the oldest kind of .spv files, which
use a
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
really funny way to encode tables.
FWIW, I'm excited that you're working on reading SPV format. I
had
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
assumed that the implicit output model might preclude
compatibility.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
-Alan
There might be some small hope here because I've been spending
a
Post by Michał Dubrawski
Post by Ben Pfaff
lot of
Post by Michał Dubrawski
time lately working to make PSPP understand SPSS's .spv output
file
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
format, so that it can read (and write?) it directly.
Implicit in
Post by Michał Dubrawski
Post by Ben Pfaff
that
Post by Michał Dubrawski
format is an understanding of the SPSS output model, which
seems
Post by Michał Dubrawski
Post by Ben Pfaff
to be
Post by Michał Dubrawski
entangled quite a bit with the CTABLES syntax. It might
therefore
Post by Michał Dubrawski
Post by Ben Pfaff
lead
Post by Michał Dubrawski
someday to CTABLES implementation (but not soon).
On Tue, Jul 24, 2018 at 09:21:11PM +0000, Crichton, Ronald
I asked about this a few years ago. I gather it would be
difficult.
Post by Michał Dubrawski
Mind you, I used TABLES in SPSS in about 1990 and got to quite
like it
Post by Michał Dubrawski
Post by Ben Pfaff
and
Post by Michał Dubrawski
used it a lot. So, here we are in 2018 and I use CTABLES in SPSS,
and
Post by Michał Dubrawski
Post by Ben Pfaff
can?t
Post by Michał Dubrawski
get the hang of it, at least not much more than the basic usage of
it.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
Can?t stand it. I long for the old TABLES function. Not available
in
Post by Michał Dubrawski
Post by Ben Pfaff
SPSS.
Post by Michał Dubrawski
So, here?s the thing: perhaps PSPP should develop its own version
of
Post by Michał Dubrawski
Post by Ben Pfaff
TABLES
Post by Michał Dubrawski
with just the basics, without trying to emulate SPSS because they
have
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
stuffed up something that used to be quite good.
From: Pspp-users [mailto:pspp-users-bounces+ronald.crichton=
Sent: Tuesday, 24 July 2018 6:29 PM
Subject: PSPP CTABLES
Hi, I would like to ask if there is any progression in CTABLES
implementation to PSPP. I read some threads (http://pspp-users.gnu
.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
narkive.com/r32JwdwI/what-features-of-custom-tables-
ctables-are-most-important) but they seems dead. Any suggestion?
Kind regards
Jacob
------------------------------------------------------------
-----------
This email, and any attachments, may be confidential and also
privileged. If you are not the intended recipient, please notify
the
Post by Michał Dubrawski
Post by Ben Pfaff
sender
Post by Michał Dubrawski
and delete all copies of this transmission along with any
attachments
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
immediately. You should not copy or use it for any purpose, nor
disclose
Post by Michał Dubrawski
its contents to any other person.
------------------------------------------------------------
-----------
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
--
Alan D. Mead, Ph.D.
President, Talent Algorithms Inc.
science + technology = better workers
http://www.alanmead.org
"You're an interesting species. An interesting mix.
You're capable of such beautiful dreams, and such
horrible nightmares. You feel so lost, so cut off,
so alone, only you're not. See, in all our
searching, the only thing we've found that makes
the emptiness bearable, is each other."
-- Carl Sagan, Contact
Ben Pfaff
2018-07-26 23:06:36 UTC
Permalink
Thank you for the correction. I don't have a corpus of .spo or .spw
files. (If anyone has a collection of either kind of file and is
willing to give them to me, then I can make reading those formats a
later project. I would keep them confidential.)

GPL graphics are similar, except for syntax, to the ViZML graphics that
.spv files embed for graphs and models, so if I ever implement one of
them then the other would probably not be much harder. But I don't
currently have plans for either one.
Post by Michał Dubrawski
Thank you Ben,
I remember the old output format from my first contact with SPSS as a
student - I found information that it was .spo file extension while .spw is
https://stats.idre.ucla.edu/spss/seminars/introduction-to-spss-syntax-2/
" SPSS output files have the file extension .spv. You can also save output
files with the .spw extension, which is for the SPSS Web Reports format.
Older SPSS output files may have the extension .spo. These files can be
read with the SPSS Smart Viewer, which can be downloaded for free from the
IBM SPSS website. Note that only .spo files that were created using a
Windows machine can be read with the SPSS Smart Viewer. "
https://www.ibm.com/support/knowledgecenter/en/SSLVMB_24.0.0/spss/base/syn_begin_gpl_overview.html
"GPL blocks cannot be contained within DEFINE-!ENDDEFINE macro definitions"
I don't really like that GPL command language, but I don't know if you ever
plan to have graphics like that in PSPP.
warm regards,
Michal
Post by Ben Pfaff
There seem to be several SPSS output formats.
First, there's something really old (?) with an .spw extension. I
haven't looked at those at all and I don't have a corpus of them to
learn from, so PSPP won't support .spw files until those change.
Second, there are effectively two different subformats of .spv files.
Both of them wrap XML and binary members inside a .zip file. In the
older format, graphs and tables are both represented by ViZML XML. This
is a fairly awful way to represent a table, so the newer format switches
to a binary format for tables (SPSS calls this format "light"). There
are different sub-versions for that binary format, too. Currently my
reader understands the light table format (all versions I've run into)
and I'm working on the older format.
I haven't tried to make PSPP understand SPSS graphs. That's a lot more
work.
When this feature gets in, it's probably going to take several
iterations of bug reports and fixes, because I'm implementing it by
reading lots of .spv files and making sure that the reader supports
them. Even though it successfully reads the thousands of .spv files
I've collected, I imagine that there will be lots that it initially
cannot. For those, I hope that users will send them to me so that I can
add them to my corpus and fix the bugs. Other than those bugs, PSPP
should be able to open any .spv file, regardless of what procedures they
use.
Which graphics commands can't be called from macros?
Thanks,
Ben.
Post by Michał Dubrawski
Thank you Ben. I understand the context now. Good to hear that spv format
will help. By the way does it mean that unlike SPSS which if I remember
well cannot open spv created in older versions of SPSS, PSPP would be
able
Post by Michał Dubrawski
to open any spv? Would it be only able to read spv with output from
procedures that are also implemented in PSPP?
One more thing I really hate how they made graphics commends which could
not be called from macros - please don't go that way if you would like to
implement it in PSPP someday :)
Post by Ben Pfaff
MULT RESPONSE might also benefit from the work I'm doing. I've looked
at MULT RESPONSE before, and one of the things that kept me from
implementing it is the output complexity. CROSSTABS also has very
complex output routines, to the extent the complexity of the output
comes close to overshadowing the complexity of the data analysis. MULT
RESPONSE is worse.
But most of the complexity comes from how PSPP internally has no
systematic approach to displaying data as tables. (It does have decent
support for rendering tables into different formats, but not for
converting data into tables to be rendered.) The SPV format actually
points a good way forward on this point, so when the appropriate amount
of work is done, it should allow CROSSTABS to be nicer internally and
then a reasonable MULT RESPONSE implementation too.
Post by Michał Dubrawski
Hi Ben,
it is great to hear that you are working with spv output. I know that
they
Post by Michał Dubrawski
were changing it from version to version so it seem like quite a
challenge,
Post by Michał Dubrawski
but based on what challenges you have faced before when making PSPP
compatible to SPSS I'm sure you will handle it :)
CTABLES or Tables in the future will be very useful but I think that
what
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
would be most important for students, social sicentists,
psychologists,
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
market researchers or virtualy anyone who analyse any survey results
would
Post by Michał Dubrawski
be the possibility to display mrsets in one table, which could be
done in
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
SPSS with MULT RESPONSE command. Without it PSPP wouldn't be an
option
Post by Michał Dubrawski
Post by Ben Pfaff
to
Post by Michał Dubrawski
display survey results because almost all surveys have multi response
questions. The only free alternative would be some packages in R
which
Post by Michał Dubrawski
Post by Ben Pfaff
are
Post by Michał Dubrawski
not as accessible (easy to use) as PSPP.
Any way to display MRSETS in the PSPP output would be very valuable
for a
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
lot of people even if it would be a cross tabs extension allowing the
usage
Post by Michał Dubrawski
of mrsets in it or something like that. When people graduate from
psychology, sociology, social science or pedagogics departments they
have
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
knowledge how to build a surveys - there are a lot of tools that
allow
Post by Michał Dubrawski
Post by Ben Pfaff
them
Post by Michał Dubrawski
to collect data via online surveys, but when it comes to displaying
the
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
results in tables for the reports they all have a software problem
because
Post by Michał Dubrawski
many non-profits and startups cannot afford SPSS + Custom Tables
licenses,
Post by Michał Dubrawski
and from my experience that is exactly the software they know how to
use
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
because of their Universities.
That said, thanks again to Ben and all the developers and all other
people
Post by Michał Dubrawski
involved with PSPP project - you have done tremendous work so far.
Michał Dubrawski,
Date: Wed, 25 Jul 2018 08:48:33 -0700
​​
Re: PSPP CTABLES
Content-Type: text/plain; charset=utf-8
I think it's likely that I'll change PSPP to use the same output
model.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
One reason that it differed until now is simply that it wasn't
clear to
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
me how SPSS really modeled output. The documentation on it is
pretty
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
muddy.
I've now got code that reads and understands most .spv files just
fine.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
I'm still working on reading the oldest kind of .spv files, which
use a
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
really funny way to encode tables.
FWIW, I'm excited that you're working on reading SPV format. I
had
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
assumed that the implicit output model might preclude
compatibility.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
-Alan
There might be some small hope here because I've been spending
a
Post by Michał Dubrawski
Post by Ben Pfaff
lot of
Post by Michał Dubrawski
time lately working to make PSPP understand SPSS's .spv output
file
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
format, so that it can read (and write?) it directly.
Implicit in
Post by Michał Dubrawski
Post by Ben Pfaff
that
Post by Michał Dubrawski
format is an understanding of the SPSS output model, which
seems
Post by Michał Dubrawski
Post by Ben Pfaff
to be
Post by Michał Dubrawski
entangled quite a bit with the CTABLES syntax. It might
therefore
Post by Michał Dubrawski
Post by Ben Pfaff
lead
Post by Michał Dubrawski
someday to CTABLES implementation (but not soon).
On Tue, Jul 24, 2018 at 09:21:11PM +0000, Crichton, Ronald
I asked about this a few years ago. I gather it would be
difficult.
Post by Michał Dubrawski
Mind you, I used TABLES in SPSS in about 1990 and got to quite
like it
Post by Michał Dubrawski
Post by Ben Pfaff
and
Post by Michał Dubrawski
used it a lot. So, here we are in 2018 and I use CTABLES in SPSS,
and
Post by Michał Dubrawski
Post by Ben Pfaff
can?t
Post by Michał Dubrawski
get the hang of it, at least not much more than the basic usage of
it.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
Can?t stand it. I long for the old TABLES function. Not available
in
Post by Michał Dubrawski
Post by Ben Pfaff
SPSS.
Post by Michał Dubrawski
So, here?s the thing: perhaps PSPP should develop its own version
of
Post by Michał Dubrawski
Post by Ben Pfaff
TABLES
Post by Michał Dubrawski
with just the basics, without trying to emulate SPSS because they
have
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
stuffed up something that used to be quite good.
From: Pspp-users [mailto:pspp-users-bounces+ronald.crichton=
Sent: Tuesday, 24 July 2018 6:29 PM
Subject: PSPP CTABLES
Hi, I would like to ask if there is any progression in CTABLES
implementation to PSPP. I read some threads (http://pspp-users.gnu
.
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
narkive.com/r32JwdwI/what-features-of-custom-tables-
ctables-are-most-important) but they seems dead. Any suggestion?
Kind regards
Jacob
------------------------------------------------------------
-----------
This email, and any attachments, may be confidential and also
privileged. If you are not the intended recipient, please notify
the
Post by Michał Dubrawski
Post by Ben Pfaff
sender
Post by Michał Dubrawski
and delete all copies of this transmission along with any
attachments
Post by Michał Dubrawski
Post by Ben Pfaff
Post by Michał Dubrawski
immediately. You should not copy or use it for any purpose, nor
disclose
Post by Michał Dubrawski
its contents to any other person.
------------------------------------------------------------
-----------
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
_______________________________________________
Pspp-users mailing list
https://lists.gnu.org/mailman/listinfo/pspp-users
--
Alan D. Mead, Ph.D.
President, Talent Algorithms Inc.
science + technology = better workers
http://www.alanmead.org
"You're an interesting species. An interesting mix.
You're capable of such beautiful dreams, and such
horrible nightmares. You feel so lost, so cut off,
so alone, only you're not. See, in all our
searching, the only thing we've found that makes
the emptiness bearable, is each other."
-- Carl Sagan, Contact
Loading...