Discussion:
[freetds] Handling type, new and old
Frediano Ziglio
2014-01-20 15:23:53 UTC
Permalink
Hi,
as discussed recently adding a new type to libTDS is quite a nightmare!

I was thinking a different way to handle our types. Mainly something
similar to functions pointers we have but extended so support
everything. I would like to extend it in a way that every upper
library could have additional "methods" specific to this library.
However I want to not have to code of a specific library in libTDS.
libTDS should be able to get the table of a specific type with the
extended information and call method it needs. Only upper layer will
see additional methods. I'd like that when a new type is added to
libTDS compilation of upper layer will fail until type is correctly
supported. Also I'd like something that does not require many
allocation but I would prefer statically allocated structures. My
implementation idea is this:

libTDS has an array of pointer to structures indexed by type to the
function pointers, something like:

struct tds_type_pointers type_funcs[] = {
...
funcs_chars,
...
funcs_int,
};

Now, libTDS will provide implementations for functions but NOT for the
structures. That way to avoid link failures upper layers has to
provide the structures, better if they extend them. So for instance
ODBC could define an extended structure

struct odbc_type_pointers {
tds_type_pointers common;
unsigned (*get_sql_type)(TDSCOLUM *col);
};

...

struct odbc_type_pointers funcs_chars = {
{ tds_char_get, ...},
odbc_get_char_sql_type;
}

Does we agree on doing that. Probably it will require some changes (a
lot) but adding a new type will be very easy! Probably will require
some additional macro and a lot of definitions but beside that we
avoid to miss some pieces.

Frediano
Frediano Ziglio
2014-01-20 15:51:03 UTC
Permalink
Post by Frediano Ziglio
Hi,
as discussed recently adding a new type to libTDS is quite a nightmare!
I was thinking a different way to handle our types. Mainly something
similar to functions pointers we have but extended so support
everything. I would like to extend it in a way that every upper
library could have additional "methods" specific to this library.
However I want to not have to code of a specific library in libTDS.
libTDS should be able to get the table of a specific type with the
extended information and call method it needs. Only upper layer will
see additional methods. I'd like that when a new type is added to
libTDS compilation of upper layer will fail until type is correctly
supported. Also I'd like something that does not require many
allocation but I would prefer statically allocated structures. My
libTDS has an array of pointer to structures indexed by type to the
struct tds_type_pointers type_funcs[] = {
...
funcs_chars,
...
funcs_int,
};
Now, libTDS will provide implementations for functions but NOT for the
structures. That way to avoid link failures upper layers has to
provide the structures, better if they extend them. So for instance
ODBC could define an extended structure
struct odbc_type_pointers {
tds_type_pointers common;
unsigned (*get_sql_type)(TDSCOLUM *col);
};
...
struct odbc_type_pointers funcs_chars = {
{ tds_char_get, ...},
odbc_get_char_sql_type;
}
Does we agree on doing that. Probably it will require some changes (a
lot) but adding a new type will be very easy! Probably will require
some additional macro and a lot of definitions but beside that we
avoid to miss some pieces.
Some example why. Let's assume libTDS wants to add a new type. For
instance libTDS does not handle tables (yes, mssql 2012 can return a
table as a type!). We add a declaration (but NO a definition) for a
funcs_table. Now we try to compile and links fails as linker does not
fund funcs_table definition (actually it fails in dblib, ctlib and
odbc). We have to define funcs_table in every upper library.

Frediano
Marc Abramowitz
2014-01-20 16:57:22 UTC
Permalink
Or another example is DATE and TIME.

Periodically, pymssql users ask why they can use a Python datetime but not a date or time.

I would guess this questions comes up for other language bindings too.

-Marc
http://marc-abramowitz.com
Sent from my iPhone 4S
Post by Frediano Ziglio
Post by Frediano Ziglio
Hi,
as discussed recently adding a new type to libTDS is quite a nightmare!
I was thinking a different way to handle our types. Mainly something
similar to functions pointers we have but extended so support
everything. I would like to extend it in a way that every upper
library could have additional "methods" specific to this library.
However I want to not have to code of a specific library in libTDS.
libTDS should be able to get the table of a specific type with the
extended information and call method it needs. Only upper layer will
see additional methods. I'd like that when a new type is added to
libTDS compilation of upper layer will fail until type is correctly
supported. Also I'd like something that does not require many
allocation but I would prefer statically allocated structures. My
libTDS has an array of pointer to structures indexed by type to the
struct tds_type_pointers type_funcs[] = {
...
funcs_chars,
...
funcs_int,
};
Now, libTDS will provide implementations for functions but NOT for the
structures. That way to avoid link failures upper layers has to
provide the structures, better if they extend them. So for instance
ODBC could define an extended structure
struct odbc_type_pointers {
tds_type_pointers common;
unsigned (*get_sql_type)(TDSCOLUM *col);
};
...
struct odbc_type_pointers funcs_chars = {
{ tds_char_get, ...},
odbc_get_char_sql_type;
}
Does we agree on doing that. Probably it will require some changes (a
lot) but adding a new type will be very easy! Probably will require
some additional macro and a lot of definitions but beside that we
avoid to miss some pieces.
Some example why. Let's assume libTDS wants to add a new type. For
instance libTDS does not handle tables (yes, mssql 2012 can return a
table as a type!). We add a declaration (but NO a definition) for a
funcs_table. Now we try to compile and links fails as linker does not
fund funcs_table definition (actually it fails in dblib, ctlib and
odbc). We have to define funcs_table in every upper library.
Frediano
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
Frediano Ziglio
2014-01-20 18:03:54 UTC
Permalink
Well... at least DATE/TIME are half way supported. Tables are
completely not supported.
By the way... do you like my idea/proposal?

Frediano
Post by Marc Abramowitz
Or another example is DATE and TIME.
Periodically, pymssql users ask why they can use a Python datetime but not a date or time.
I would guess this questions comes up for other language bindings too.
-Marc
http://marc-abramowitz.com
Sent from my iPhone 4S
Post by Frediano Ziglio
Post by Frediano Ziglio
Hi,
as discussed recently adding a new type to libTDS is quite a nightmare!
I was thinking a different way to handle our types. Mainly something
similar to functions pointers we have but extended so support
everything. I would like to extend it in a way that every upper
library could have additional "methods" specific to this library.
However I want to not have to code of a specific library in libTDS.
libTDS should be able to get the table of a specific type with the
extended information and call method it needs. Only upper layer will
see additional methods. I'd like that when a new type is added to
libTDS compilation of upper layer will fail until type is correctly
supported. Also I'd like something that does not require many
allocation but I would prefer statically allocated structures. My
libTDS has an array of pointer to structures indexed by type to the
struct tds_type_pointers type_funcs[] = {
...
funcs_chars,
...
funcs_int,
};
Now, libTDS will provide implementations for functions but NOT for the
structures. That way to avoid link failures upper layers has to
provide the structures, better if they extend them. So for instance
ODBC could define an extended structure
struct odbc_type_pointers {
tds_type_pointers common;
unsigned (*get_sql_type)(TDSCOLUM *col);
};
...
struct odbc_type_pointers funcs_chars = {
{ tds_char_get, ...},
odbc_get_char_sql_type;
}
Does we agree on doing that. Probably it will require some changes (a
lot) but adding a new type will be very easy! Probably will require
some additional macro and a lot of definitions but beside that we
avoid to miss some pieces.
Some example why. Let's assume libTDS wants to add a new type. For
instance libTDS does not handle tables (yes, mssql 2012 can return a
table as a type!). We add a declaration (but NO a definition) for a
funcs_table. Now we try to compile and links fails as linker does not
fund funcs_table definition (actually it fails in dblib, ctlib and
odbc). We have to define funcs_table in every upper library.
Frediano
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
Marc Abramowitz
2014-01-20 18:08:40 UTC
Permalink
What do you mean by "half way supported"? (I think pymssql might not be
taking advantage of whatever support is there).

Love the idea of creating a framework that makes it easier to add types.
Clearly it's difficult to add a type with the current system, so if it can
be made more straightforward, that is a great step!
Post by Frediano Ziglio
Well... at least DATE/TIME are half way supported. Tables are
completely not supported.
By the way... do you like my idea/proposal?
Frediano
Post by Marc Abramowitz
Or another example is DATE and TIME.
Periodically, pymssql users ask why they can use a Python datetime but
not a date or time.
Post by Marc Abramowitz
I would guess this questions comes up for other language bindings too.
-Marc
http://marc-abramowitz.com
Sent from my iPhone 4S
Post by Frediano Ziglio
Post by Frediano Ziglio
Hi,
as discussed recently adding a new type to libTDS is quite a
nightmare!
Post by Marc Abramowitz
Post by Frediano Ziglio
Post by Frediano Ziglio
I was thinking a different way to handle our types. Mainly something
similar to functions pointers we have but extended so support
everything. I would like to extend it in a way that every upper
library could have additional "methods" specific to this library.
However I want to not have to code of a specific library in libTDS.
libTDS should be able to get the table of a specific type with the
extended information and call method it needs. Only upper layer will
see additional methods. I'd like that when a new type is added to
libTDS compilation of upper layer will fail until type is correctly
supported. Also I'd like something that does not require many
allocation but I would prefer statically allocated structures. My
libTDS has an array of pointer to structures indexed by type to the
struct tds_type_pointers type_funcs[] = {
...
funcs_chars,
...
funcs_int,
};
Now, libTDS will provide implementations for functions but NOT for the
structures. That way to avoid link failures upper layers has to
provide the structures, better if they extend them. So for instance
ODBC could define an extended structure
struct odbc_type_pointers {
tds_type_pointers common;
unsigned (*get_sql_type)(TDSCOLUM *col);
};
...
struct odbc_type_pointers funcs_chars = {
{ tds_char_get, ...},
odbc_get_char_sql_type;
}
Does we agree on doing that. Probably it will require some changes (a
lot) but adding a new type will be very easy! Probably will require
some additional macro and a lot of definitions but beside that we
avoid to miss some pieces.
Some example why. Let's assume libTDS wants to add a new type. For
instance libTDS does not handle tables (yes, mssql 2012 can return a
table as a type!). We add a declaration (but NO a definition) for a
funcs_table. Now we try to compile and links fails as linker does not
fund funcs_table definition (actually it fails in dblib, ctlib and
odbc). We have to define funcs_table in every upper library.
Frediano
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
Stephen Marshall
2014-01-20 18:41:54 UTC
Permalink
Freddy,

I think your idea to consolidate type-support code is great. On general
principles, I am in favor of refactoring that makes it easier to support
and extend any code base.

My only concern is if the benefit is worth the effort. If there are still
a number of unsupported data types in FreeTDS, the rework would be worth
it. However, if DATE is one of the few remaining unsupported types, it
might be more cost (and time) effective to shoe-horn them into the existing
framework.

To figure out which way to go, it might be worth assessing what data types
are currently NOT supported. Off the top of my head, it appears the
geographic data types in MSSQL (geometry and geography) fall in the
unsupported category. I'm not sure if there are others.

Yours,
Steve
Post by Marc Abramowitz
What do you mean by "half way supported"? (I think pymssql might not be
taking advantage of whatever support is there).
Love the idea of creating a framework that makes it easier to add types.
Clearly it's difficult to add a type with the current system, so if it can
be made more straightforward, that is a great step!
On Mon, Jan 20, 2014 at 10:03 AM, Frediano Ziglio <freddy77 at gmail.com
Post by Frediano Ziglio
Well... at least DATE/TIME are half way supported. Tables are
completely not supported.
By the way... do you like my idea/proposal?
Frediano
Post by Marc Abramowitz
Or another example is DATE and TIME.
Periodically, pymssql users ask why they can use a Python datetime but
not a date or time.
Post by Marc Abramowitz
I would guess this questions comes up for other language bindings too.
-Marc
http://marc-abramowitz.com
Sent from my iPhone 4S
On Jan 20, 2014, at 7:51 AM, Frediano Ziglio <freddy77 at gmail.com>
Post by Frediano Ziglio
Post by Frediano Ziglio
Hi,
as discussed recently adding a new type to libTDS is quite a
nightmare!
Post by Marc Abramowitz
Post by Frediano Ziglio
Post by Frediano Ziglio
I was thinking a different way to handle our types. Mainly something
similar to functions pointers we have but extended so support
everything. I would like to extend it in a way that every upper
library could have additional "methods" specific to this library.
However I want to not have to code of a specific library in libTDS.
libTDS should be able to get the table of a specific type with the
extended information and call method it needs. Only upper layer will
see additional methods. I'd like that when a new type is added to
libTDS compilation of upper layer will fail until type is correctly
supported. Also I'd like something that does not require many
allocation but I would prefer statically allocated structures. My
libTDS has an array of pointer to structures indexed by type to the
struct tds_type_pointers type_funcs[] = {
...
funcs_chars,
...
funcs_int,
};
Now, libTDS will provide implementations for functions but NOT for
the
Post by Frediano Ziglio
Post by Marc Abramowitz
Post by Frediano Ziglio
Post by Frediano Ziglio
structures. That way to avoid link failures upper layers has to
provide the structures, better if they extend them. So for instance
ODBC could define an extended structure
struct odbc_type_pointers {
tds_type_pointers common;
unsigned (*get_sql_type)(TDSCOLUM *col);
};
...
struct odbc_type_pointers funcs_chars = {
{ tds_char_get, ...},
odbc_get_char_sql_type;
}
Does we agree on doing that. Probably it will require some changes (a
lot) but adding a new type will be very easy! Probably will require
some additional macro and a lot of definitions but beside that we
avoid to miss some pieces.
Some example why. Let's assume libTDS wants to add a new type. For
instance libTDS does not handle tables (yes, mssql 2012 can return a
table as a type!). We add a declaration (but NO a definition) for a
funcs_table. Now we try to compile and links fails as linker does not
fund funcs_table definition (actually it fails in dblib, ctlib and
odbc). We have to define funcs_table in every upper library.
Frediano
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
Frediano Ziglio
2014-01-21 22:03:19 UTC
Permalink
Well,
the start is something like
https://gitorious.org/freetds/mars-freetds/commit/2a490d7cd88a6f093135910dc4292a8e6eb412b9.

Actually I'm trying it (running tests against).

The next would to to move other type handling spread all around to
some extended function pointers. Actually the structures can also
contains some data if necessary, probably the only constraint is that
they must be constant.

Frediano
Post by Stephen Marshall
Freddy,
I think your idea to consolidate type-support code is great. On general
principles, I am in favor of refactoring that makes it easier to support
and extend any code base.
My only concern is if the benefit is worth the effort. If there are still
a number of unsupported data types in FreeTDS, the rework would be worth
it. However, if DATE is one of the few remaining unsupported types, it
might be more cost (and time) effective to shoe-horn them into the existing
framework.
To figure out which way to go, it might be worth assessing what data types
are currently NOT supported. Off the top of my head, it appears the
geographic data types in MSSQL (geometry and geography) fall in the
unsupported category. I'm not sure if there are others.
Yours,
Steve
Post by Marc Abramowitz
What do you mean by "half way supported"? (I think pymssql might not be
taking advantage of whatever support is there).
Love the idea of creating a framework that makes it easier to add types.
Clearly it's difficult to add a type with the current system, so if it can
be made more straightforward, that is a great step!
On Mon, Jan 20, 2014 at 10:03 AM, Frediano Ziglio <freddy77 at gmail.com
Post by Frediano Ziglio
Well... at least DATE/TIME are half way supported. Tables are
completely not supported.
By the way... do you like my idea/proposal?
Frediano
Post by Marc Abramowitz
Or another example is DATE and TIME.
Periodically, pymssql users ask why they can use a Python datetime but
not a date or time.
Post by Marc Abramowitz
I would guess this questions comes up for other language bindings too.
-Marc
http://marc-abramowitz.com
Sent from my iPhone 4S
On Jan 20, 2014, at 7:51 AM, Frediano Ziglio <freddy77 at gmail.com>
Post by Frediano Ziglio
Post by Frediano Ziglio
Hi,
as discussed recently adding a new type to libTDS is quite a
nightmare!
Post by Marc Abramowitz
Post by Frediano Ziglio
Post by Frediano Ziglio
I was thinking a different way to handle our types. Mainly something
similar to functions pointers we have but extended so support
everything. I would like to extend it in a way that every upper
library could have additional "methods" specific to this library.
However I want to not have to code of a specific library in libTDS.
libTDS should be able to get the table of a specific type with the
extended information and call method it needs. Only upper layer will
see additional methods. I'd like that when a new type is added to
libTDS compilation of upper layer will fail until type is correctly
supported. Also I'd like something that does not require many
allocation but I would prefer statically allocated structures. My
libTDS has an array of pointer to structures indexed by type to the
struct tds_type_pointers type_funcs[] = {
...
funcs_chars,
...
funcs_int,
};
Now, libTDS will provide implementations for functions but NOT for
the
Post by Frediano Ziglio
Post by Marc Abramowitz
Post by Frediano Ziglio
Post by Frediano Ziglio
structures. That way to avoid link failures upper layers has to
provide the structures, better if they extend them. So for instance
ODBC could define an extended structure
struct odbc_type_pointers {
tds_type_pointers common;
unsigned (*get_sql_type)(TDSCOLUM *col);
};
...
struct odbc_type_pointers funcs_chars = {
{ tds_char_get, ...},
odbc_get_char_sql_type;
}
Does we agree on doing that. Probably it will require some changes (a
lot) but adding a new type will be very easy! Probably will require
some additional macro and a lot of definitions but beside that we
avoid to miss some pieces.
Some example why. Let's assume libTDS wants to add a new type. For
instance libTDS does not handle tables (yes, mssql 2012 can return a
table as a type!). We add a declaration (but NO a definition) for a
funcs_table. Now we try to compile and links fails as linker does not
fund funcs_table definition (actually it fails in dblib, ctlib and
odbc). We have to define funcs_table in every upper library.
Frediano
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
Loading...